Improvements to RucioFileCatalogClient, compatibility of older client with Rucio 41 server

[maxnoe]

BEGINRELEASENOTES

• FIX: RucioFileCatalog when using rucio clients <41 with rucio server >= 41
  ENDRELEASENOTES

This makes a couple of small updates to the RucioFileCatalogClient triggered by issues we discovered in the CTAO integration test environment when running DIRAC with a rucio client version 40 but a rucio server version 41.

In Rucio 41, a long-standing bug ignoring the plugin parameter for get_metadata_bulk was fixed, which in prior versions resulted in the DID_COLUMN metadata being returned regardless of the plugin= parameter.

However, this breaks the RFC in the combination of a rucio 40 client with a rucio 41 server (a combination which is allowed according to Rucio's support policy of newer servers keeping compatibility with older clients), because get_metadata_bulk in rucio 40 uses plugin='JSON" as the default. This worked before because plugin was ignored, however, with a Rucio 41 server, plugin is correctly respected, which results in the JSON metadata being returned where the RFC actually expects the DID_COLUMN metadata (e.g. containing size in bytes, checksums, did type).

Using Rucio client 41 with Rucio server 41 works, because also the default plugin for get_metadata_bulk was changed to DID_COLUMN.

Here, it is better to be explicit though: everywhere DIRAC needs the basic did metadata, DID_COLUMN is explicitly added.

I also changed a couple of calls of get_did_metadata to get_did, which is cheaper to execute as it does not involve the metadata plugin machinery.

[maxnoe]

@cserf

[cserf]

LGTM. The PR is marked as draft, so do you want to had more changes ?

[maxnoe]

I left it draft for now because I didn't come around to testing all changes yet in our integration environment.

I tested most and it works fine, but e.g. the data removal methods I didn't test yet as they are not convered in our integration test suite.