Before embarking on a large scale data migration from a legacy system into a new system, it is important to establish a game plan.  Part of the game plan is to profile the data that will be migrated.  Unfortunately, too many times I see Data Analyst diving into the source database and begin writing extensive SQL queries that provide various permutations of data profiling reports before determining if the data is even relevant to the new system.   I believe that by having a game plan for data profiling and subsequently, establishing a remediation plan, the organization will save a lot of time and money.

This is a repost of the attached article first published in the PMI Virtual Library.

In the consulting world, project estimation is a critical component required for the delivery of a successful project.  If you estimate correctly, you will deliver a project on time and within budget; get it wrong and you could end up over budget, with an unhappy client and a burned out team.  Project estimation for business intelligence and data integration projects is especially difficult, given the number of stakeholders involved across the organization as well as the unknowns of data complexity and quality.  Add to this mix a firm fixed price RFP (request for proposal) response for a client your organization has not done work for and you have the perfect climate for a poor estimate.  In this article, I share my thoughts about the best way to approach a project estimate for an extract, transform load (ETL) project.