refactor: adapt ResourceStatus/SiteStatus consumers to binary model by HeloiseJoffe · Pull Request #8738 · DIRACGrid/DIRAC

HeloiseJoffe · 2026-08-12T08:58:20Z

BEGINRELEASENOTES
*ResourceStatusSystem
CHANGE: adapt ResourceStatus/SiteStatus consumers to binary model
ENDRELEASENOTES

fstagni · 2026-08-25T08:05:37Z

Hi, this one LGTM. At the same time there are some failing tests in the backward compatibility:

https://github.com/DIRACGrid/DIRAC/actions/runs/31604149176/job/94138661798?pr=8738#step:13:359 :

LocalRepo/TestCode/DIRAC/tests/Integration/ResourceStatusSystem/Test_ResourceStatus.py::test_getElementStatus FAILED [100%]

=================================== FAILURES ===================================
____________________________ test_getElementStatus _____________________________

rssClient = <DIRAC.ResourceStatusSystem.Client.ResourceStatusClient.ResourceStatusClient object at 0x7f532e5434d0>
rsClient = <DIRAC.ResourceStatusSystem.Client.ResourceStatus.ResourceStatus object at 0x7f532e47a610>

    def test_getElementStatus(rssClient: ResourceStatusClient, rsClient):
        # make sure that the test resoureces are not presented in the db
        rssClient.deleteStatusElement("Resource", "Status", "testActiveResource")
        rssClient.deleteStatusElement("Resource", "Status", "testBannedResource")
        rssClient.deleteStatusElement("Resource", "Status", "testResource")
    
        res = rssClient.insertStatusElement(
            "Resource",
            "Status",
            "testActiveResource",
            "all",
            "Degraded",
            "ComputingElement",
            "reason",
            Datetime,
            Datetime,
            "tokenOwner",
            Datetime,
        )
    
        assert res["OK"] is True, res["Message"]
        rsClient.rssCache.refreshCache()
    
        res = rsClient.getElementStatus("testActiveResource", "ComputingElement")
    
        assert res["OK"] is True, res["Message"]
>       assert res["Value"]["testActiveResource"]["all"] == "Active"
E       AssertionError: assert 'Degraded' == 'Active'
E         
E         - Active
E         + Degraded

and https://github.com/DIRACGrid/DIRAC/actions/runs/31604149176/job/94138661798?pr=8738#step:13:491

LocalRepo/TestCode/DIRAC/tests/Integration/ResourceStatusSystem/Test_SiteStatus.py::test_addAndRemove_complicatedTest FAILED [100%]

=================================== FAILURES ===================================
______________________ test_addAndRemove_complicatedTest _______________________

stClient = <DIRAC.ResourceStatusSystem.Client.SiteStatus.SiteStatus object at 0x7fa47b1ff710>

    def test_addAndRemove_complicatedTest(stClient):
        rsClient = ResourceStatusClient()
        result = rsClient.insertStatusElement(
            "Site",
            "Status",
            "testActive.test.test",
            "all",
            "Active",
            "Site",
            "Synchronized",
            Datetime,
            Datetime,
            "tokenOwner",
            Datetime,
        )
        assert result["OK"] is True, result["Message"]
    
        result = rsClient.insertStatusElement(
            "Site",
            "Status",
            "testActive1.test.test",
            "all",
            "Active",
            "Site",
            "Synchronized",
            Datetime,
            Datetime,
            "tokenOwner",
            Datetime,
        )
        assert result["OK"] is True, result["Message"]
    
        result = rsClient.insertStatusElement(
            "Site",
            "Status",
            "testBanned.test.test",
            "all",
            "Banned",
            "Site",
            "Synchronized",
            Datetime,
            Datetime,
            "tokenOwner",
            Datetime,
        )
        assert result["OK"] is True, result["Message"]
        stClient.rssCache.refreshCache()
    
        # TEST getSites
        # ...............................................................................
    
        result = stClient.getSites()
        assert result["OK"] is True, result["Message"]
        inRSS = "testActive.test.test" in result["Value"]
    
        # TEST getSites
        # ...............................................................................
    
        result = stClient.getSites("all")
        assert result["OK"] is True, result["Message"]
    
        # TEST getUsableSites
        # ...............................................................................
    
        result = stClient.getUsableSites()
        assert result["OK"] is True, result["Message"]
    
        # setting a status
        if inRSS:
            result = stClient.setSiteStatus("testActive.test.test", "Probing")
            assert result["OK"] is True, result["Message"]
            stClient.rssCache.refreshCache()
    
            result = stClient.getSites("Banned")
            assert result["OK"] is True, result["Message"]
>           assert "testActive.test.test" in result["Value"]
E           AssertionError: assert 'testActive.test.test' in ['testBanned.test.test']

LocalRepo/TestCode/DIRAC/tests/Integration/ResourceStatusSystem/Test_SiteStatus.py:157: AssertionError

These are testing that a client of DIRAC version v9.0.X can talk with a server with version v9.1.X (this version).

aldbr

There are other places that would need to be checked:

The if, elif blocks should likely disappear (we should not gate the writing part anymore):

DIRAC/src/DIRAC/DataManagementSystem/scripts/dirac_admin_ban_se.py

Lines 116 to 125 in 36806a1

    
           if seOptions["ReadAccess"] == "Banned": 
        
               gLogger.notice("Read access already banned", se) 
        
               resR["OK"] = True 
        
           elif not seOptions["ReadAccess"] in ["Active", "Degraded", "Probing", "Error"]: 
        
               gLogger.notice( 
        
                   "Read option for %s is %s, instead of %s" 
        
                   % (se, seOptions["ReadAccess"], ["Active", "Degraded", "Probing", "Error"]) 
        
               ) 
        
               gLogger.notice("Try specifying the command switches") 
        
           else:

DIRAC/src/DIRAC/DataManagementSystem/scripts/dirac_admin_ban_se.py

Lines 135 to 144 in 36806a1

    
           if write and "WriteAccess" in seOptions: 
        
               if seOptions["WriteAccess"] == "Banned": 
        
                   gLogger.notice("Write access already banned", se) 
        
                   resW["OK"] = True 
        
               elif not seOptions["WriteAccess"] in ["Active", "Degraded", "Probing"]: 
        
                   gLogger.notice( 
        
                       "Write option for %s is %s, instead of %s" 
        
                       % (se, seOptions["WriteAccess"], ["Active", "Degraded", "Probing"]) 
        
                   ) 
        
                   gLogger.notice("Try specifying the command switches")

DIRAC/src/DIRAC/DataManagementSystem/scripts/dirac_admin_ban_se.py

Lines 155 to 165 in 36806a1

    
           if check and "CheckAccess" in seOptions: 
        
               if seOptions["CheckAccess"] == "Banned": 
        
                   gLogger.notice("Check access already banned", se) 
        
                   resC["OK"] = True 
        
               elif not seOptions["CheckAccess"] in ["Active", "Degraded", "Probing"]: 
        
                   gLogger.notice( 
        
                       "Check option for %s is %s, instead of %s" 
        
                       % (se, seOptions["CheckAccess"], ["Active", "Degraded", "Probing"]) 
        
                   ) 
        
                   gLogger.notice("Try specifying the command switches") 
        
               else:

Same here:

DIRAC/src/DIRAC/DataManagementSystem/scripts/dirac_admin_allow_se.py

Lines 125 to 136 in 36806a1

    
           if statusFlagDict[statusType]: 
        
               if seOptions.get(statusType) == "Active": 
        
                   gLogger.notice(f"{statusType} status of {se} is already Active") 
        
                   continue 
        
               if statusType in seOptions: 
        
                   if not seOptions[statusType] in ALLOWED_STATUSES: 
        
                       gLogger.notice( 
        
                           "%s option for %s is %s, instead of %s" 
        
                           % (statusType, se, seOptions["ReadAccess"], ALLOWED_STATUSES) 
        
                       ) 
        
                       gLogger.notice("Try specifying the command switches") 
        
                   else:

aldbr · 2026-08-27T06:27:43Z

        if not (result := self.getSiteMask(status="Active"))["OK"]:
            return result
        siteMask = result["Value"]
        if site in siteMask:
            if printOutput:
                gLogger.notice(f"Site {site} is already Active")
            return S_OK(f"Site {site} is already Active")


Suggested change

This check should be removed I think.
A site in a Degraded status is seen as Active on the reading side and therefore if an admin wants to make it Active for whatever reason, it should likely not be prevented.

aldbr · 2026-08-27T06:30:20Z

        if site in siteMask:
            if printOutput:
                gLogger.notice(f"Site {site} is already Banned")
            return S_OK(f"Site {site} is already Banned")

        tokenLifetime = int(days)
        if tokenLifetime <= 0:
            tokenExpiration = datetime.max


Suggested change

Same suggestion here, an admin might want to set Banned if the status is Probing

aldbr · 2026-08-27T06:33:46Z

+        if entry[1] in ALLOWED:
+            status = "Active"
+        else:
+            status = "Banned"


This piece of code appears twice (also in ResourceStatus) exactly the same way and could reused after in #8733.
I would extract it in a small utility function, in PolicySystem/StateMachine may be?

aldbr · 2026-08-27T06:36:15Z

+            status = "Active"
+        else:
+            status = "Banned"
+        res.update({(entry[0]): status})


Isn't it the same as:

Suggested change

res.update({(entry[0]): status})

res[entry[0]] = status

aldbr · 2026-08-27T06:37:56Z


        return cacheMatch

    def getUsableSites(self, siteNames=None):


Just wondering: isn't it equivalent to getSites("Active") now?
What about just calling it inside getUsableSites?

aldbr · 2026-08-27T06:39:08Z

            S_OK( { 'SE1': { 'ReadAccess': 'Probing' ,
                             'WriteAccess': 'Active',
                             'CheckAccess': 'Degraded',
                             'RemoveAccess': 'Banned'} } } )


Suggested change

S_OK( { 'SE1': { 'ReadAccess': 'Banned' ,

'WriteAccess': 'Active',

'CheckAccess': 'Active',

'RemoveAccess': 'Banned'} } } )

HeloiseJoffe requested review from atsareg and fstagni as code owners August 12, 2026 08:58

HeloiseJoffe force-pushed the refactor/dirac_consumers_to_binary_status_model branch from a080078 to 873af57 Compare August 12, 2026 13:57

HeloiseJoffe marked this pull request as draft August 13, 2026 06:05

HeloiseJoffe requested a review from aldbr August 25, 2026 06:26

aldbr requested changes Aug 27, 2026

View reviewed changes

refactor: adapt ResourceStatus/SiteStatus singletons cache to binary

c209b8a

aldbr force-pushed the refactor/dirac_consumers_to_binary_status_model branch from 873af57 to c209b8a Compare August 28, 2026 09:29

Navigation Menu

Uh oh!

refactor: adapt ResourceStatus/SiteStatus consumers to binary model - #8738

refactor: adapt ResourceStatus/SiteStatus consumers to binary model#8738
HeloiseJoffe wants to merge 1 commit into
DIRACGrid:integrationfrom
HeloiseJoffe:refactor/dirac_consumers_to_binary_status_model

HeloiseJoffe commented Aug 12, 2026

Uh oh!

fstagni commented Aug 25, 2026

Uh oh!

aldbr left a comment

Uh oh!

aldbr Aug 27, 2026

Uh oh!

aldbr Aug 27, 2026

Uh oh!

aldbr Aug 27, 2026

Uh oh!

aldbr Aug 27, 2026

Uh oh!

aldbr Aug 27, 2026

Uh oh!

aldbr Aug 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	if seOptions["ReadAccess"] == "Banned":
	gLogger.notice("Read access already banned", se)
	resR["OK"] = True
	elif not seOptions["ReadAccess"] in ["Active", "Degraded", "Probing", "Error"]:
	gLogger.notice(
	"Read option for %s is %s, instead of %s"
	% (se, seOptions["ReadAccess"], ["Active", "Degraded", "Probing", "Error"])
	)
	gLogger.notice("Try specifying the command switches")
	else:

	if write and "WriteAccess" in seOptions:
	if seOptions["WriteAccess"] == "Banned":
	gLogger.notice("Write access already banned", se)
	resW["OK"] = True
	elif not seOptions["WriteAccess"] in ["Active", "Degraded", "Probing"]:
	gLogger.notice(
	"Write option for %s is %s, instead of %s"
	% (se, seOptions["WriteAccess"], ["Active", "Degraded", "Probing"])
	)
	gLogger.notice("Try specifying the command switches")

	if check and "CheckAccess" in seOptions:
	if seOptions["CheckAccess"] == "Banned":
	gLogger.notice("Check access already banned", se)
	resC["OK"] = True
	elif not seOptions["CheckAccess"] in ["Active", "Degraded", "Probing"]:
	gLogger.notice(
	"Check option for %s is %s, instead of %s"
	% (se, seOptions["CheckAccess"], ["Active", "Degraded", "Probing"])
	)
	gLogger.notice("Try specifying the command switches")
	else:

	if statusFlagDict[statusType]:
	if seOptions.get(statusType) == "Active":
	gLogger.notice(f"{statusType} status of {se} is already Active")
	continue
	if statusType in seOptions:
	if not seOptions[statusType] in ALLOWED_STATUSES:
	gLogger.notice(
	"%s option for %s is %s, instead of %s"
	% (statusType, se, seOptions["ReadAccess"], ALLOWED_STATUSES)
	)
	gLogger.notice("Try specifying the command switches")
	else:

Uh oh!

Conversation

HeloiseJoffe commented Aug 12, 2026

Uh oh!

fstagni commented Aug 25, 2026

Uh oh!

aldbr left a comment

Choose a reason for hiding this comment

Uh oh!

aldbr Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

aldbr Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

aldbr Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

aldbr Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

aldbr Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

aldbr Aug 27, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants