Now that I’ve spent some time describing the various benefits of using metadata-driven design, I think that it’s probably time to finally expound on the process via a working example. After all, it doesn’t seem like a concrete method until it has been put into practice. So, as I had described in a previous post, my group here at Barnes & Noble needed a new product data system and all of its inherent constituents (databases, applications, etc.). However, before we started the project, we needed to define its scope and its functionality, and in this case, domain-driven design assisted in the creation of a manifest for our requirements. However, since this system had never been created with such a focus on foresight and since we were all new to architecture design, we didn’t even know how to start this phase of the design process. It became frustrating when we tried to simply converse about a particular point of the project, and documentation with images and text didn’t seem to help the situation much.
For example, all of the stakeholders agreed that the system should know how to handle each data point via a number of dimensions: retrieving/persisting, validating, applying permissions, auditing, error handling, etc. In order to discuss how we would create a design that accounted for all of these specifications, we needed to put something on the whiteboard that we could all scrutinize together. Since the managers and directors were not as familiar with programming, the use of UML, pseudocode, and prototyped classes did not resonate with them. However, to some degree, it did seem that everyone was familiar with databases and SQL. So, a few of us decided to draw some hypothetical tables on the whiteboard and populate them with data, which would give shape to the items listed in the manifest. Suddenly, the room became alive as the whiteboard became a sounding board, and people began molding this table data that described our requirements. After that initial meeting, we created the tables on the whiteboard in a separate database, and everyone obtained access to them. We then went through multiple iterations of people altering the schema’s tables, conversing through emails about those changes, and then having another rendezvous to talk about them on the whiteboard.
Soon, this simple iteration began to produce results. Initially, the information about the data points’ dimensions (to which we began referring as metadata) was organized into appropriate metadata tables:
In one case, discussions quickly came to a consensus that in some cases, permissions should not be applicable to only one data point. For example, a product’s price was actually a set of data points: its numerical value, its type (retail, cost, etc.), its currency, etc. In that case, an user authorized to change price data should be able to change all of those data points, and the system should be intelligent enough to group the permissions for such a set of related data points. So, in an effort to create our own version of an access control list, we then took the metadata about permissions and altered it to reflect this new enhancement to our system:
It’s somewhat difficult to emphasize its psychological impact, but this emergent productivity from creating a few simple tables excited us as we successfully began to build an architecture through small increments. Through only a few simple iterations of this metadata, we had quickly understood our permissions problem and drafted a decent approach to resolving the issue. At this point, we were only thinking of it as a design tool, not as a potential foundation layer for our architecture. However, as we repeatedly reshaped this metadata to better fit our desired solution, someone eventually pondered with the question “I wonder if we could build something that just ran off of these tables?” To which, a few of us nodded in cautious, pensive optimism…but that’s a story for another day.