Loading results¶
Core
report_analyst/ ยท Reciprocal Public License. This is the open core: the Streamlit analyst, chunking, question sets, and benchmarking. It ships in the core Docker image.
These loaders turn a CSV or SQLite export into a benchmark dataset. ClimRetrieve files are detected and rewritten into query_id / chunk_id / position / score before scoring.
report_analyst.core.benchmark.retrieval_results_loader.load_flexible_dataset_from_csv(csv_path=None, csv_content=None, dataset_id=None, dataset_name=None, dataset_type=None, column_mapping=None)
¶
Load a flexible benchmark dataset from CSV with automatic column name detection.
This function is webhook-ready and handles different column name variations. It automatically detects dataset type (IR vs IE) and maps columns flexibly.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
csv_path
|
Optional[str]
|
Path to CSV file (if loading from file) |
None
|
csv_content
|
Optional[Union[str, bytes]]
|
CSV content as string or bytes (if loading from upload/webhook) |
None
|
dataset_id
|
Optional[str]
|
Optional dataset identifier (auto-generated if not provided) |
None
|
dataset_name
|
Optional[str]
|
Optional dataset name |
None
|
dataset_type
|
Optional[DatasetType]
|
Optional dataset type (auto-detected if not provided) |
None
|
column_mapping
|
Optional[Dict[str, str]]
|
Optional explicit column mapping (standard_name -> actual_name) |
None
|
Returns:
| Type | Description |
|---|---|
BenchmarkDataset
|
BenchmarkDataset with loaded results |
Raises:
| Type | Description |
|---|---|
ValueError
|
If neither csv_path nor csv_content is provided |
FileNotFoundError
|
If csv_path is provided but file doesn't exist |
report_analyst.core.benchmark.retrieval_results_loader.detect_dataset_type(columns)
¶
Detect dataset type based on column names.
Information Retrieval (IR) datasets typically have: - chunk_id, position, score, similarity_score
Information Extraction (IE) datasets typically have: - answer, analysis, category, confidence_score
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
columns
|
List[str]
|
List of column names in the dataset |
required |
Returns:
| Type | Description |
|---|---|
DatasetType
|
DatasetType enum value |
report_analyst.core.benchmark.retrieval_results_loader.load_retrieval_results_from_csv(csv_path=None, csv_content=None, dataset_id=None, dataset_name=None)
¶
Load retrieval results from CSV file or content.
This function is webhook-ready and can accept either a file path or CSV content.
Expected CSV format: - query_id: Question/query identifier (required) - report_id: Report/document identifier (optional) - chunk_id: Chunk identifier (required) - chunk_text: Chunk text content (optional) - position: Position/rank in retrieval (required, 1-indexed) - score: Retrieval score (required) - similarity_score: Vector similarity score (optional) - llm_score: LLM-based relevance score (optional)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
csv_path
|
Optional[str]
|
Path to CSV file (if loading from file) |
None
|
csv_content
|
Optional[Union[str, bytes]]
|
CSV content as string or bytes (if loading from upload/webhook) |
None
|
dataset_id
|
Optional[str]
|
Optional dataset identifier (auto-generated if not provided) |
None
|
dataset_name
|
Optional[str]
|
Optional dataset name (defaults to filename or 'uploaded_dataset') |
None
|
Returns:
| Type | Description |
|---|---|
RetrievalResultsDataset
|
RetrievalResultsDataset with loaded results |
Raises:
| Type | Description |
|---|---|
ValueError
|
If neither csv_path nor csv_content is provided |
FileNotFoundError
|
If csv_path is provided but file doesn't exist |