Bob Lambert
Apr 30 2013
Data Governance Begins At the Spreadsheet
Data management professionals have long and sometimes rather Quixotically driven organizations to “get past the spreadsheet culture.” Maybe that’s misguided. The recent furor over a widely read social science paper may show how we can look to scientific peer review for a way to govern data, spreadsheets and all.
Recently, it was found that a key study underpinning debt-reduction as a driver of economic growth based its conclusions on a flawed spreadsheet. As this ArsTechnica article describes, Carmen Reinhart and Kenneth Rogoff's Growth in a Time of Debt seemingly proved a connection between "high levels of debt and negative average economic growth". But, per a recent study by Thomas Herndon, Michael Ash, and Robert Pollin, it turns out that the study's conclusions drew from a Microsoft Excel formula mistake, questionable data exclusions, and non-standard weightings of base data. The ArsTechnica piece finds those conclusions fade to a more ambiguous outcome with errors and apparent biases corrected.
Mar 26 2013
How to Criticize Your Boss
Not everyone gets to be a leader, and most leaders are also followers in their own right. The project manager follows instructions from the project sponsor, the CEO from the board, the politicians from the polls, and so on. Whoever you are, you spend a lot more time following than leading. As Bob Dylan put it so well, "you gotta serve somebody."
The good follower is not a "yes man". In the professional world I inhabit those who move "up" the hierarchy tend to retire technical skills in favor of architecture, proposal writing, and management. The relationship of manager to employee becomes more like agent to actor or musician, where the supervised employee is the "talent".
In these conditions the old concept of top-down decision making seems quaint. Important choices require information from all perspectives, and organizations shut out those with knowledge of the details at their peril. The best decision makers search out diverse ideas before choosing a direction.
Feb 26 2013
A Fist Full of Agile Critiques
Out of curiosity I recently reviewed articles critical of Agile Methodologies. I had expected agile-versus-waterfall arguments and attacks from vendors selling new alternatives,
but even given the reputation that advocates have for flaming well-intentioned critics, I wasn’t prepared for the level of emotion I found.
My opening position was that Agile techniques are great, but like any other tool there are limits and prerequisites. The critical articles I read strengthened that view. Let’s review three examples that stood out, in reverse order:
Jan 25 2013
Relational DB Pros: The Times They Are A-Changin
Recently I read a thoughtful post at the PASS Business
Analytics Conference site discussing how different the world is now for database professionals. Author Chris Webb focuses on the data science side in this post. His analysis made me think of the challenges and opportunities "big data" serves up to relational database designers.
To me these challenges are fundamental. Big Data and NoSQL bring lots of what we know about data elements, inherent data design, and data management into question. I think considering these elements closely leads to a sensible to-do list for relational database professionals.
Jan 08 2013
Data Design Matters
As important as it is, data modeling has always had a geeky, faintly impractical tinge to some.
I've seen application development projects proceed with a suboptimal, "good enough", model. The resulting systems might otherwise be well-architected, but sometimes strange vulnerabilities emerge that track directly to data design flaws
Recently I saw an example where a "good enough" data design, similar to the one pictured, enabled a significant application bug.