File management

Upload

Edison Analysis is designed to run data analysis on files provided by the user or caller. To provide Edison Analysis with this data, you'll need to upload it to the Edison data storage service. This service is your one stop shop for sharing, storing and updating data to be used in the Edison ecosystem.

To upload a single file to the data storage service:

single_file_upload_response = await client.astore_file_content(
    name="Demo file entry for a single file",
    file_path="./datasets/brain_size_data.csv",  # ADD DATASET PATH HERE
    description="This is a test file that will be be analysed by Edison Analysis",
)

To upload a directory to the data storage service:

directory_upload_response = await client.astore_file_content(
    name="Demo file entry for a whole directory",
    file_path="./datasets",  # ADD DATASET FOLDER PATH HERE
    description="This is a directory that will be be analysed by Edison Analysis",
    as_collection=True,
)

Download

While the task is executing it will create some artifacts. First the notebook which is where the analysis code will be written and any other artifacts creating during the task.

Once the task has completed you may want to check the contents of the notebook or look through the artifacts generated. To obtain these artifacts, you will need to inspect the output of the agent's final environment_frame

output_data = job_result.environment_frame["state"]["info"]["output_data"]
print(output_data)

for output_file in output_data:
    download_response = await client.afetch_data_from_storage(
        data_storage_id=output_file["entry_id"]
    )

    # Note there are two potential outcomes here. One where the client downloads
    # the file to your local filesystem if it's above ~10MB. The second is where
    # it will return a RawFetchResponse object which contains the raw content.
    print(download_response)