Wednesday, March 21, 2007

Data Warehousing

I am reading this article.

Being in the storage industry, I am trying to understand how companies store and use data. Data warehouses are huge silos of data and are used for Business Intelligence (BI).

The technical definition is something like this:

"A database designed to support decision making in an organization. Data from the production databases are copied to the data warehouse so that queries can be performed without disturbing the performance or the stability of the production systems. Data warehouses can become enormous with hundreds of gigabytes of transactions. As a result, subsets, known as "data marts," are often created for just one department or product line. The data warehouse is structured to support a variety of analyses, including elaborate queries on large amounts of data that can require extensive searching. When databases are set up for queries on daily transactions, they are often called "operational data stores" rather than data warehouses."

This suggests that these databases are point in time snapshots of the production database. This would allow BI software to track changes in things like inventory size. At some point data in the depot becomes old and irrelevant and is perhaps deleted.