Data Reliability – A Critical Requirement for Integration

During the 1980s and 1990s, almost every major company went through some form of re-engineering and restructuring, as thousands of workers and managers were shed in a variety of efforts to increase productivity and reduce costs. In conjunction with these efforts, organizations implemented information systems to perform tasks previously performed by these workers. They installed systems such as Material Requirements Planning (MRP), Distribution Requirements Planning (DRP), and its broader application, Enterprise Resource Planning (ERP). Worries about the Year 2000 (Y2K) bug, required careful analysis of these “legacy” systems to determine if any computer programming bugs existed as companies entered the new millennium.

No sooner was this over than a new challenge loomed: restructuring businesses to function in the new era of electronic commerce. Presented with a deluge of information on “dot-coms,” servers, business-to-business (B2B) requirements, and on-line customer and supplier linkages, organizations are now struggling to catch up to this new business requirement. At the same time, organizations continue to reduce staffing levels and managers must face the challenge of performing at higher levels with fewer resources. For many organizations, the key to improved productivity and decision-making is the expanded use of information systems. Such systems can help to improve the internal integration of business functions. Information systems along with the web technology is required for improvement of internal and external business functions. Integration needs to be with internal as well as external systems. This is where the major problems are: integration of heterogeneous systems built over several years now are either replaced by these new systems or need to be data sources/sinks for the new systems. One supply chain software implementation expert noted that “in my experience, 70-80% of cost and effort in application deployment and BPR is in integration. Within this area, major problems are encountered in verification and validation of data.” The result: hours and hours of manual labor required to “cleanse” the data and render it usable.

Internal integration of business information systems is a critical step in SCM. Why? Because internal integration enables the monolithic task of integrating one’s customers and suppliers. Unless one can communicate information effectively between internal business functions, information flows with external supply chain partners are unlikely to succeed. As one manager we interviewed said, “We need to be able to improve our own business processes and link them to our databases. Otherwise, we’re simply pumping our garbage over the Internet to our customers and suppliers!” Indeed, the problem of internal data integration and data validity continues to be a significant problem for organizations in almost every industry.

The biggest issue companies struggle with in this area is data quality and data collection. The essential problem is the need to get down to detailed “data dictionary” levels in order to define specific business processes. Recently, a consulting company hosted a session to assign business process events to specific information system terms. The group spent more than two hours defining the term “on-time arrival!” Before deploying any B2B system, companies must begin at the basic level described here, define their business processes carefully, then assign specific meaning to data elements based on events known to occur within these business processes. Data integration is becoming increasingly problematic for companies attempting to link their information systems with customers and suppliers; even big retailers such as Wal-Mart generally communicate orders in free-form text, with no master data dictionary. Fundmental basic standards for criteria such as time zones are often overlooked: EDI/XML has a defined time zone, but few organizations use it, and all carriers frequently utilize different time zones with little commonality between them. Companies must begin with this level of detail before tackling larger “visionary” B2B supply chain issues. This lesson is one of the most difficult for the “dot-com” era, when companies purchased new B2B software based on fuzzy objectives for the project, and without fully comprehending the need for data standards. When these customers realized the Herculean effort required to implement these systems, the money suddenly stopped flowing; so did the stock prices of the software vendors who sold them.