In my second professional article, I describe how metadata-driven design can be employed to build an ingestion system for data retrieved through an API. I must give many thanks to the staff at InfoQ for taking the time to help with this submission. If you have any interest or stake in enterprise-level data ingestion, there’s a good chance that you’ll find something of value in it. Go check it out!
Metadata-Driven Design: Designing a Flexible Engine for API Data Retrieval
Bulk data is commonly accessed via files & FTP. As the world moves toward APIs to facilitate collaboration, what are the requirements for data APIs? In this article, Aaron Kendall of Barnes & Noble describes a meta-data driven architecture for bulk data ingestion. Two APIs operate in parallel to provide data changes as well as the data records themselves. An example demonstrates how API responses are parameterized using meta-data.