jump to navigation

Own your Data 11/29/2010

Posted by TBoehm30 in Data.
Tags: , , , ,
add a comment

“There is too much data at my company to be useful.”  “The data we have is old and out-of-date.”  “Users around here don’t enter good data.”  How often are these statements said or thought?

Data is important and should be controlled properly.  The only way to ensure that you have good data is to make someone responsible for it, and have them own the data.  They need to use the data as well as have authority over the information put into the database.  Only when someone relies on data will they take an active part in making sure that the data is valid.

I once had a meeting with users who didn’t like the process of putting in a caller’s first name and last name in the text boxes provide.  They wanted to put all of the caller’s information into the notes section.  I was supposed to figure out how to make that work.  You can’t report on notes, you can’t sort on them, you can’t aggregate them, they are practically useless as a management tool.  I had to tell the users to get in line with the processes of the day and fill in the caller’s information in the proper location on the screen.

Do you have documentation on how data flows through your company?  Do you know where your data comes from?  How good is your data?  How often is it refreshed?  Do you have duplicate information?  What about duplicate sources of information?  If somebody came to me and told me that a report was wrong, I can trace it back to the source to figure out the problem.  Maybe he is looking at an old report, maybe the data is coming from the wrong columns, and maybe a data point or two are bad.  Whatever the problem, I know how to find the original data, determine the refresh rate, evaluate the quality of the data and explain any variance.

Once data gets old, it becomes difficult to validate.  You will need to de-dup your data, or find and remove any duplicates.  This is impossible if you don’t have good data.  How will you know if 2 entries with the same name are the same person or different people who just happen to have similar names?  I don’t know many people who can go through thousands of records to find duplicates and finish without going crazy.  If you find someone like that, keep them around.

Software exists that can do a lot of the validation for you.  You might have to give up control of your data to use it; or it might cost a lot of money.  The last time I looked at that kind of software they gave us percentage rates on the validity of the results.  I wouldn’t use the cheap stuff that had less than an 80% chance of being right.

I once worked with a guy who had to de-dup a huge database.  He explained that very few people with the same last name were born on the same day.  So, if you had their last name and birth date, you could be reasonably certain of duplicates.  Of course, twins and multiples are a problem.  In that case you add in the first name and it is extremely rare to find duplicates that were in fact more than one person.

He told me about twins with the same first name, but different middle names.  Why would a parent do that?  He told me about a time when he had a name with two addresses and two Social Security Numbers that were one digit off.  He just knew that the SSN was a typo for one record, but he couldn’t prove it.  He had to list the two records as two people and it was bugging him.  So, he drove by the guy’s house to see if the addresses were correct (he wasn’t allowed to contact anyone because of privacy).  Sure enough the addresses were on a corner and looked to be the same house.  What could he do about it?  Nothing – he was not allowed to fix the data.

What sort of data issues do you wish you could fix at your company?  Do the right people own the data and take care of it?  Do you have governance to control the quality of the information?

Companies need to value their information, validate it often, and use it to their advantage.  After all, it’s a global world out there, and Technology makes it happen.

Test Before Installing an Upgrade 10/29/2010

Posted by TBoehm30 in Software.
Tags: , , , , , , , , , ,
add a comment

A new risk to your software is coming; it is time for an upgrade.  Remember when your system was new?  Remember that huge project to implement the software?  Now, it seems, you don’t know how you survived without it.  Don’t take unnecessary risks; make sure you are prepared for this upgrade.

An upgrade may just be a small change, it could be a large change, it could be a few patches, it could be a brand new user interface, and it could contain brand new modules.  No matter the size, you want to make sure that this upgrade doesn’t break any critical processes before you install it.  Even if you get experts to put it in, you will want to do some testing first.

There are several ways of getting an upgrade installed.  You can get the software company to do it for you.  This can be included in your support contract, or you might pay extra for it.  You could hire a consultant who is an expert on the software you are using.  If you are going this route it is best when you have a working relationship with the consultants so that they understand your business as well as the specific software.  You can also do it yourself.  Many software companies provide instructions for how to put in the new code.

The first step, however, is always going to be testing.  A good software company will test the upgrades themselves.  The reason you pay money for the software support is so that you can trust the software will do what it is supposed to.  If the software doesn’t do what you want, you can call the company for help.  A good consultant will schedule in plenty of testing before an upgrade.  They should know what the upgrade is meant to improve, and with a good understanding of your company, should know what to test.  Of course, if you do this yourself, then testing should be second nature.

There are several areas to test before upgrading.  Any customization to the software should be at the top of the list.  Since you changed the default behavior of the program, you hope that the underlying logic didn’t change.  If it did, you will need to either remove your customization, or re-create the custom code on the new version of the software.

The next areas to test require a little bit of study.  Usually the software company will publish documentation on the changes.  You should review those changes and mark any functionality that is critical.  Sometimes the documentation can be technical and confusing; if that is the case, simply write down the module, area, functionality, or process that needs to be tested. 

Last on the list of testing is any other area you think should be tested before an upgrade.  I have seen people just do a full validation before any upgrade. 

During the implementation phase for this software, you documented your business processes.  Hopefully, you had a documented test script.  Save those!  If you have good test scripts, they can be used again for this process.  If you don’t have test scripts, then make them now and you can use them for each upgrade.  Spend some time on those scripts, they are valuable.  If you have a good set of test scripts, then you understand your system; you have the ability to validate your system and your data.

Make a list of all the functionalities that are important to you.  Start out with broad categories such as A/P, A/R, Manufacturing, or CRM.  Then narrow the focus to specifics such as returning merchandise, or receiving a complaint from a customer.  If you document the steps you take for each of these specifics, and make screen shots of the results, you are on your way to a great set of test scripts.

Before letting anyone upgrade your software system, make sure you back up all of your data.  Make sure you can restore yourself to the previous version in case catastrophe strikes. 

If you have properly tested the new software, then you shouldn’t run into any surprises.  Remember, it’s a global world out there and Technology makes it happen.

Spin Off 07/23/2010

Posted by TBoehm30 in Project Management.
Tags: , , , , , , , ,
add a comment

How does a company chop it’s IT in order to spin off and sell a part of itself?

I just read an excellent article by Mike Gorsage from Tatum about Integrating IT for Mergers and Acquisitions.  In the second half of the article he talks about “Planning for Separation.”

 This got me thinking about when I was consulting with Agilent as they were getting ready to split the company and sell part of it to Philips.  Agilent had just recently split from HP and some of the IT was still, unofficially, shared.  While HP and Agilent were two different companies, there was a shared culture that allowed people to communicate easily.  People at Agilent had come from HP and were comfortable with how things worked there.  Now some of them were being ‘sold’ to Philips and it had to change.

 Agilent had to figure out what IT the new company would take with it.  Some hardware and software would have to be duplicated, some would have to be given, and some would not be part of the sale.  They had to figure out the same thing with the IT team; however it is much more difficult to duplicate a person.

 I was working on their CRM system for the call center dealing with medical machinery.  HP/Agilent (now Philips) produced monitor type equipment for hospitals and doctors.  When the equipment had problems, they called the response center in Atlanta.  The call center would specifically be part of the sale to Philips.

 The CRM system, Clarify (now owned by Amdocs), was highly integrated with the back office system.  Corporate Agilent decided that the back office system would not be part of the sale.  Since Philips and Agilent, along with HP, are competitors in some arenas, the new company could have no access to proprietary data.

 That meant we had to get all of our data off of the old system, into a new system on a very tight timeline.  We had to update our CRM system to integrate with SAP instead of the old HP system.  With the help of some very smart people, we got the CRM system up to speed within our timeline and on budget.  We figured out how to download master data from SAP: the customers, contacts, products, and contracts.  We figured out how to send back to SAP the data it needed for tracking, billing, etc. 

 During the building move we managed to move all of the hardware over the weekend, so barely anyone knew it happened.  The call center is a 24 by 7 operation, so it had to be done carefully and quickly.

 The biggest problem turned out to be migrating data from the back office system to the new SAP system.  The contracts in the legacy system were confusing, not standardized, and big.  Agilent had a large team of people who came up with a plan to pull the data, clean it up, and then put it into SAP.  It would take weeks of processing the data after it was pulled, to clean it up enough to get into SAP.

 Agilent set up the deadline of when to be out of their system.  If the new company missed that deadline and needed continued access for another month it would cost Philips a lot of money.  We missed that deadline and had to extend.  Agilent doubled their fee the next month and would have doubled it again had we needed another month.

 By that time, I suppose, someone decided enough was enough, and called it done.  We had enough data to proceed and make it work.  Since I was not involved directly with old back end system, I have no idea what data we lost.  I can only imagine it was significant.

 With the new company complete, we worked on better integration with SAP, the politics of the new culture, and improving the call center.  Philips had bought several medical companies that would join us and we had to integrate them next.

 Most of the companies’ call centers were moved into our building and so we eventually shared the CRM system with them all.  We shared the phone system, and the other infrastructure that allowed us connectivity to the internet, intranet, and each other.

 Philips wanted everyone to use SAP in the call center.  We quickly did benchmarks and comparisons to show corporate how much extra it would cost to get rid of the current CRM system.  They acquiesced, at least temporarily, to allow the legacy system.  Their idea was still to get rid of it, and that theme was seen in numerous meetings.  We managed to continue the use of Clarify, even against the wishes of some big-wigs at Philips IT.  Eventually it was expanded to other countries around the world.

 The thing that Philips did right was allow the best technology to continue being used, and even invested in its future.  While they had alternate plans, the old systems worked well and could not easily be changed so they didn’t change it.  Their alternate plans were shelved, and new plans created to better the company as a whole.

 Philips had the people who understand it’s a global world out there and Technology makes it happen.

How to Decide on Customizations 05/13/2009

Posted by TBoehm30 in Software.
Tags: , , , , ,
add a comment

Customizing your new software is one of the dangers that can lead to the project spiraling out of control. Too many customized changes or changes that are too complicated will alter the schedule, bog down the staff with work, and lower moral.

I worked with a company that installed a new CRM system with only the vanilla, out-of-the-box screens. The software had great tools for customizations and allowed all sorts of changes without affecting the license, or even the upgrade process. If the company followed the rules, then the upgrade process would take into account the customizations, and not cause problems. The company decided not to allow any requests for changes for the first 3 months.

This turned out to be a terrific rule. Many of the users complained because the system didn’t look like their old system. They weren’t sure what to do when problems occurred. During those 3 months, the users got used to the new system. They figured out how to solve their own problems. They found the workarounds for any issues that were serious enough to change their process.

After the 3 months were up, the changes started coming in. Those changes were well thought out, and would provide value to the business. The changes requested were the kind that IT could get working on, and wouldn’t hinder existing projects. They had new timelines for the new projects, so nobody felt like they were falling behind. The budgets were identified separately for the new projects, so no projects were over budget.

The IT department looked great because their large software project was successfully completed on time and on budget. They surveyed the business managers to find their satisfaction with the project. They got high marks for their project management abilities and their ability to get people working on the new system. The new requested changes would be worked on with full approval from management who have faith in their IT people. This confidence was gained through successful projects, and satisfied users. What a great working environment this was.

I have also worked on projects that were overwhelmed by changes. Our weekly status meetings showed that we should be aiming for a moving target. The work we had finished the previous week was now obsolete and we had to start over. Some of the requests (in our opinion) were absolutely ridiculous. Some of them were most likely submitted for the sole purpose of sabotaging the project. Many of the users were very afraid that the new system would make their job much harder. We ended up scuttling the project because we had no chance for success.

Somewhere in the middle of these two extremes has to be a good model. Users will have legitimate changes to a project while it is underway. Sometimes the project managers won’t have thought of all the consequences until users start testing. Sometimes the users don’t understand the consequences of their changes. Sometimes neither side will understand how complicated it becomes to change large software systems.

The takeaway – create a process to evaluate customization requests for your software. Understand the costs and the benefits of those changes before deciding on if and when to implement. Remember that it’s a global world out there and technology makes it happen.

Scrubbing the Data 04/07/2009

Posted by TBoehm30 in Data.
Tags: , , , ,
1 comment so far

Data needs to be ‘clean’ in a database to be worth anything. How can management trust their reports if their previous decisions turned out wrong based on bad data?

My favorite story on fixing the data was one from a seriously good techie named Mike. Every month or so, several departments would get called out based on the percentage of errors in the system. They got called on silly things like end dates before start dates, check boxes not checked when required, etc. These would have been easy to constrain if they had the ability to do that to the system. Unfortunately they were not allowed to make changes to their back end system.

Mike put in a new CRM system as a front end, and put in controls to prevent the users from entering ‘stupid’ data. This effectively gave them a 0% error rate. The next month Mike got called out because management didn’t believe the data. They assumed something was wrong with the data since it had no errors. That just wasn’t possible.

Not only was it possible, but it made Mike’s department the model for the rest of the company. His (Our) CRM system started to become the new front end for other departments that could use it. We made it a practice to put good business logic anywhere that data could get into our system.

The question then becomes: Where does it make the most sense to scrub the data? If you have control of 1 system, then any place where data goes into that system is an opportunity to clean the data. On user’s screens is the obvious location. Don’t let users enter mistakes into the system. Correct them before the data is saved. For example, if you have a dropdown choice, and require a checkbox for one of the choices, then put that in code. Even better, if the checkbox needs to be checked only 80% of the time, but half the time, the user forgets, then make sure they get a prompt reminding them of the checkbox.

Not so visible entry points include interfaces. Interfaces push, pull, send, or receive data from other systems. You may not have control over those other systems. Creating one set of APIs or controls is ideal. You should use the same set of requirements for any data coming into the system. Then you need a process where data can be fixed when it is found to be in error.

When an error is found, there are several options for what to do. You can route the data to a human, you could send an error back to the original system and not accept the data at all, or you could flag the data for future follow-up. What you shouldn’t do is allow the bad data to enter without any notice that it is wrong. You need your business logic to scrub the data for problems and inconsistencies.

If you have control over more than 1 system, then you may have more than just a few entry points. This project gets more difficult as you find more problem points with your systems. Obviously, any connection points between your systems need to be clean. Any outside connection to any of your systems need to have the data scrubbed.

Over the short run, having an end date before a begin date is not a big deal. In the long run, however, it can completely screw up an analysis. Not having the correct documentation for a sale could be the final straw pushing a manager’s dashboard control into the red and causing him to make bad decisions.

Business managers need to look at their simple errors and work with IT to improve their business logic. They need to be aware that it’s a global world out there and Technology makes it happen.