Docs
Python API
Adapter API
LoaderAdapter

LoaderAdapter

LoaderAdapter handles data loading and automatically processes benchmark:// protocol for benchmark dataset and schema file downloads.

Class Architecture 

classDiagram

    class LoaderAdapter {
        +config: dict
        +loader: Loader
        +benchmarker: Benchmarker
        +__init__(config)
        +run() tuple[DataFrame, Schema]
        -_handle_benchmark_protocol()
        -_init_loader()
    }

    class Loader {
        +filepath: str
        +schema: Schema
        +load() tuple[DataFrame, Schema]
    }

    class Benchmarker {
        +config: BenchmarkerConfig
        +download()
        +get_filepath()
    }

    class BenchmarkerConfig {
        +benchmark_name: str
        +filepath_raw: str
    }

    class BenchmarkerRequests {
        +download()
    }

    LoaderAdapter ..> Loader : uses for data loading
    LoaderAdapter ..> Benchmarker : uses for benchmark protocol
    Benchmarker --> BenchmarkerConfig : has
    Benchmarker ..> BenchmarkerRequests : creates

    %% Style definitions
    class LoaderAdapter {
        <<Main Class>>
    }
    style LoaderAdapter fill:#E6E6FA

    class Loader {
        <<Core Module>>
    }
    style Loader fill:#4169E1,color:#fff

    class Benchmarker {
        <<Benchmark Handler>>
    }
    style Benchmarker fill:#9370DB,color:#fff

    style BenchmarkerConfig fill:#FFE4E1
    style BenchmarkerRequests fill:#FFE4E1

    note for LoaderAdapter "1. Detects benchmark:// protocol\n2. Uses Benchmarker to download\n3. Uses Loader with local path"

Legend:

  • Light purple box: LoaderAdapter main class
  • Blue box: Core loading module
  • Purple box: Benchmark dataset handler module
  • Light pink box: Configuration classes
  • ..>: Dependency relationship
  • -->: Has relationship

Main Features

  • Unified interface for data loading
  • Automatic detection and handling of benchmark:// protocol for both data and schema
  • Integration of Loader and Benchmarker functionality
  • Returns data and Schema metadata
  • Supports CSV data files and YAML schema files

Method Reference

__init__(config: dict)

Initializes LoaderAdapter instance with automatic benchmark:// protocol handling.

Parameters:

  • config: dict, required
    • Configuration parameter dictionary
    • Must contain filepath key
    • Supports benchmark:// protocol
    • Optional parameters include:
      • schema: Schema file path
      • nrows: Load only specified number of rows (for quick testing)
      • delimiter, encoding, header, etc. (pandas read parameters)

run(input: dict)

Executes data loading, including automatic benchmark dataset download.

Parameters:

  • input: dict, required
    • Input parameter dictionary
    • LoaderAdapter typically receives empty dictionary {}

Returns: No direct return value. Use get_result() and get_metadata() to get results.

get_result()

Gets the loaded data.

Returns:

  • pd.DataFrame: Loaded data

get_metadata()

Gets the data’s Schema metadata.

Returns:

  • Schema: Data metadata

Usage Example 

from petsard.adapter import LoaderAdapter

# Regular file loading
adapter = LoaderAdapter({
    "filepath": "data/users.csv",
    "schema": "schemas/user.yaml"
})

# Using nrows parameter for quick testing
adapter = LoaderAdapter({
    "filepath": "data/large_dataset.csv",
    "schema": "schemas/data.yaml",
    "nrows": 1000  # Load only first 1000 rows
})

# Or using benchmark:// protocol
# adapter = LoaderAdapter({
#     "filepath": "benchmark://adult-income",
#     "schema": "benchmark://adult-income_schema"
# })

# Execute loading
adapter.run({})

# Get results
data = adapter.get_result()
metadata = adapter.get_metadata()

Workflow

  1. Protocol Detection: Check if filepath/schema uses benchmark:// protocol
  2. Benchmarker Processing (for benchmark protocol)
    • Download files locally
    • Verify SHA-256 (warning on mismatch)
    • Convert paths to local paths
  3. Data Loading: Load data and metadata

Notes

  • This is an internal API, not recommended for direct use
  • Prefer using YAML configuration files and Executor
  • Benchmark files are cached after first download
  • Results are cached until next run() call
SplitterAdapter