Note
Information in this guide corresponds to the version 39 of the Qserv REST API. Keep in mind that each implementation of the API has a specific version. The version number will change if any changes to the implementation or the API that might affect users will be made. The current document will be kept updated to reflect the latest version of the API.
The Ingest Workflow Developer’s Guide#
- Introduction
- Main concepts
- Overview of the ingest workflow
- Plan the ingest
- Prepare the input data
- Register or un-publish a database
- Register tables
- Configure the Ingest service
- Start transactions
- Figure out locations of tables and chunks
- Send the data to the workers
- Monitor the progress of the ingest activities
- Commit/abort the transactions
- Publish the database
- Verify the ingested data products
- Perform the optional post-ingest data management operation on the ingested tables
- Types of tables in Qserv
- Transactions
- Table contributions
- Publishing databases and tables
- Database families
- Overview of the ingest workflow
- Simple Workflow Example
- Advanced Scenarios
- Global configuration options
- Character sets in contributions
- Concurrency control when processing async requests
- Ingesting tables into the published catalogs
- Transaction management
- Optimizations in using the REST services
- Options for making contribution requests
- Databases with many director tables
- Ingesting ref-match tables
- Using MySQL warnings for the data quality control
- Post-Ingest Data Management Tasks
- Ingest API Reference