Hardening: simplify Avro schema resolution

automation/src/test/java/org/apache/cloudberry/pxf/automation/features/avro/HdfsReadableAvroTest.java

@@ -312,8 +312,7 @@ public void avroComplexReadSchemaFromHcfs() throws Exception {
     @Test(groups = {"features", "gpdb", "hcfs", "security"})
     public void avroComplexReadSchemaFromSegmentHosts() throws Exception {
         prepareReadableTable("avrotest_complex", AVRO_ALL_TYPES_FIELDS, hdfsPath + avroComplexFileName + SUFFIX_AVRO);
-        String schemaPath = remotePublicStage + "/" + complexAvroFile;
-        exTable.setExternalDataSchema(schemaPath);
+        exTable.setExternalDataSchema(complexAvroFile);
         gpdb.createTableAndVerify(exTable);
         // Verify results
         runSqlTest("features/hdfs/readable/avro/complex_types");

docs/access-hadoop/hdfs_avro.md

@@ -125,7 +125,9 @@ You can specify an Avro schema on both read and write operations to HDFS. You ca
 | writable   | yes | PXF uses the specified schema. |
 | writable   | no | PXF creates the Avro schema based on the external table definition. |
 
-When you provide the Avro schema file to PXF, the file must reside in the same location on each Apache Cloudberry host **or** the file may reside on the Hadoop file system. PXF first searches for an absolute file path on the Apache Cloudberry hosts. If PXF does not find the schema file there, it searches for the file relative to the PXF classpath. If PXF cannot find the schema file locally, it searches for the file on HDFS.
+When you provide the Avro schema file to PXF, the file may reside on the Hadoop file system or in the PXF classpath on every Apache Cloudberry host. For a relative schema path, PXF first searches the PXF classpath. If PXF cannot find the schema file there, or if the schema path is absolute, it resolves the path using the file system configured by `SERVER`.
+
+PXF does not open an absolute `SCHEMA` path directly on the local file system of the PXF host. It treats the path as relative to the server's `pxf.fs.basePath`, or to the configured file system root when no base path is set. For example, with `pxf.fs.basePath=/data`, `SCHEMA=/schemas/example.avsc` refers to `/data/schemas/example.avsc` on the configured file system. Schema paths must not contain parent-directory (`..`) components or the dollar sign (`$`).
 
 The `$PXF_BASE/conf` directory is in the PXF classpath. PXF can locate an Avro schema file that you add to this directory on every Apache Cloudberry host.
 
@@ -165,7 +167,7 @@ The `hdfs:avro` profile supports the following \<custom-option\>s:
 | COLLECTION_DELIM | The delimiter character(s) placed between entries in a top-level array, map, or record field when PXF maps an Avro complex data type to a text column. The default is the comma (`,`) character. (Read)|
 | MAPKEY_DELIM | The delimiter character(s) placed between the key and value of a map entry when PXF maps an Avro complex data type to a text column. The default is the colon `:` character. (Read)|
 | RECORDKEY_DELIM | The delimiter character(s) placed between the field name and value of a record entry when PXF maps an Avro complex data type to a text column. The default is the colon `:` character. (Read)|
-| SCHEMA | The absolute path to the Avro schema file on the Apache Cloudberry host or on HDFS, or the relative path to the schema file on the host. (Read and Write)|
+| SCHEMA | The path to the Avro schema file on the configured file system, resolved relative to `pxf.fs.basePath` (or the file system root), or a relative path in the PXF classpath. Absolute paths are not opened directly on the PXF host. (Read and Write)|
 | IGNORE_MISSING_PATH | A Boolean value that specifies the action to take when \<path-to-hdfs-file\> is missing or invalid. The default value is `false`, PXF returns an error in this situation. When the value is `true`, PXF ignores missing path errors and returns an empty fragment. (Read) |
 
 The PXF `hdfs:avro` profile supports encoding- and compression-related write options. You specify these write options in the `CREATE WRITABLE EXTERNAL TABLE` `LOCATION` clause. The `hdfs:avro` profile supports the following custom write options:

docs/access-objectstores/objstore_avro.md

@@ -43,6 +43,7 @@ When you read or write Avro data in an object store:
     - You must include the bucket in the schema file path. This bucket need not specify the same bucket as the Avro data file.
     - The secrets that you specify in the `SERVER` configuration must provide access to both the data file and schema file buckets.
 - The schema file path must not include spaces.
+- An absolute `SCHEMA` path is resolved in the configured object store, relative to `pxf.fs.basePath` (or the file system root). PXF does not open it directly on the local file system of the PXF host. A relative schema path is searched in the PXF classpath first.
 
 ## Creating the External Table
 

docs/administering/configuring/cfg_server.md

@@ -128,6 +128,8 @@ You can use the `pxf.fs.basePath` property to restrict a user's access to files
 
 When you configure the `pxf.fs.basePath` property for a server, PXF considers the file path specified in the `CREATE EXTERNAL TABLE` `LOCATION` clause to be relative to this base path setting, and constructs the remote path accordingly.
 
+Avro schema paths resolved in the configured file system also use this base path, including absolute paths in the `SCHEMA` option. For example, with `pxf.fs.basePath=/data`, `SCHEMA=/schemas/example.avsc` resolves to `/data/schemas/example.avsc`. A relative schema name found in the PXF classpath is loaded from the classpath instead. See [Avro Schemas and Data](../../access-hadoop/hdfs_avro.md#avro-schemas-and-data).
+
 > <b>Note:</b> You must set `pxf.fs.basePath` when you configure a PXF server for access to a network file system with a `file:*` profile. This property is optional for a PXF server that accesses a file in Hadoop or in an object store.
 
 

docs/upgrade/upgrade_pxf.md

@@ -89,3 +89,16 @@ After you install the new version of PXF, perform the following procedure:
     ``` shell
     gpadmin@coordinator$ pxf cluster start
     ```
+
+## Backward-Incompatible Changes
+
+### Migrating Avro Schemas Previously Read from Local Paths
+
+Earlier PXF releases searched the PXF host's local file system before the classpath and the configured file system. Tables that relied on this behavior must be updated. For example, `SCHEMA=/opt/pxf/schemas/example.avsc` no longer opens that file directly on the PXF host, even when the file exists.
+
+Choose one of the following migration options:
+
+- Place the schema in the file system configured by `SERVER` and update `SCHEMA` to its path relative to `pxf.fs.basePath` (or the file system root). For example, with `pxf.fs.basePath=/data`, place the schema at `/data/schemas/example.avsc` and use `SCHEMA=/schemas/example.avsc`.
+- Place the schema in `$PXF_BASE/conf` on every PXF host and use its relative classpath name. For example, place it at `$PXF_BASE/conf/schemas/example.avsc` and use `SCHEMA=schemas/example.avsc`. Synchronize the configuration with `pxf cluster sync`.
+
+These changes apply to both `.avsc` and `.avro` schema files, for read and write operations. With a `file:avro` profile, schema paths are resolved relative to the required `pxf.fs.basePath` of the file server, just like data paths.

docs/upgrade/upgrade_pxf_rpm.md

@@ -6,6 +6,8 @@ sidebar_position: 4
 
 If you installed PXF from the `rpm` or `deb` package and have configured and are using PXF in your current Apache Cloudberry installation, you must perform some upgrade actions when you install a new version of PXF.
 
+Before upgrading, review the [backward-incompatible changes](./upgrade_pxf.md#backward-incompatible-changes).
+
 The PXF upgrade procedure has two parts. You perform one procedure before, and one procedure after, you install the new version:
 
 -   [Step 1: Complete the PXF Pre-Upgrade Actions](#step-1-complete-the-pxf-pre-upgrade-actions)

server/pxf-hdfs/src/main/java/org/apache/cloudberry/pxf/plugins/hdfs/avro/AvroUtilities.java

@@ -175,6 +175,8 @@ public Object decodeString(Schema schema, String value, boolean isTopLevel, bool
     private Schema readOrGenerateAvroSchema(RequestContext context, HcfsType hcfsType, String userProvidedSchemaFile) throws IOException {
         // user-provided schema trumps everything
         if (userProvidedSchemaFile != null) {
+            // Validate before classpath lookup as well as HCFS access.
+            hcfsType.validateAndNormalizeDataSource(userProvidedSchemaFile);
             AvroSchemaFileReader schemaFileReader = schemaFileReaderFactory.getAvroSchemaFileReader(userProvidedSchemaFile);
             return schemaFileReader.readSchema(context.getConfiguration(), userProvidedSchemaFile, hcfsType, fileSearcher);
         }
@@ -300,18 +302,16 @@ private static class DefaultFileSearcher implements FileSearcher {
 
         @Override
         public File searchForFile(String schemaName) {
+            // Absolute paths belong to the configured filesystem, not the PXF host.
+            if (new File(schemaName).isAbsolute()) {
+                return null;
+            }
             try {
-                File file = new File(schemaName);
-                if (!file.exists()) {
-                    URL url = AvroUtilities.class.getClassLoader().getResource(schemaName);
-
-                    // Testing that the schema resource exists
-                    if (url == null) {
-                        return null;
-                    }
-                    file = new File(URLDecoder.decode(url.getPath(), "UTF-8"));
+                URL url = AvroUtilities.class.getClassLoader().getResource(schemaName);
+                if (url == null) {
+                    return null;
                 }
-                return file;
+                return new File(URLDecoder.decode(url.getPath(), "UTF-8"));
             } catch (UnsupportedEncodingException e) {
                 LOG.info(e.toString());
                 return null;

server/pxf-hdfs/src/main/java/org/apache/cloudberry/pxf/plugins/hdfs/avro/JsonAvroSchemaFileReader.java

@@ -16,7 +16,7 @@ public class JsonAvroSchemaFileReader implements AvroSchemaFileReader {
     @Override
     public Schema readSchema(Configuration configuration, String schemaName, HcfsType hcfsType, AvroUtilities.FileSearcher fileSearcher) throws IOException {
         InputStream schemaStream = null;
-        // try searching local disk (full path name and relative path in classpath) first.
+        // Try searching the classpath for a relative schema name first.
         // by searching locally first we can avoid an extra call to external (fs.exists())
         try {
             File file = fileSearcher.searchForFile(schemaName);