Powered by
hl7.decisions.com
INTRO If you already know what HL7 is and how it is used then jump down to the Challenges Presented by HL7 section because this intro section will just put you to sleep. If HL7 is new to you‌ read on.
B
BASICS What is HL7 ? Powered by
HL7 is a text based standard for sending and receiving health care information electronically. It supports the management, delivery, and evaluation of health services. It is the ‘language’ that most healthcare software applications use to talk to each other. It has been around since 1987, and most modern hardware and software systems in healthcare support some form or version of it. New regulations requiring systems that can interoperate and governmental programs like “meaningful use” credits are also driving more adoption. In short, HL7 is how healthcare applications talk. There are other supplemental or specialized communication standards and formats that also might be used. These sometimes are vendor specific or are dictated by groups like Medicare/Medicaid. The HL7 specification is governed by Health Level Seven International, a notfor-profit, ANSI-accredited standards developing organization.
How do HL7 systems communicate? HL7 is human readable text that is broken up into meaningful sections and sent as data packets from application to application across well-defined communication mechanisms with ‘handshaking’ and acknowledgement procedures. Because the data is sent in readable format (i.e., text that can be opened on any computer) and structured according to the HL7 specification, and because the communication mechanisms are well known, HL7 tends to work well within most system infrastructures.
B
BASICS HL7 Lingo
L
ike any technology, HL7 uses a glossary of specific terms that have specific meaning. While an interface engine alleviates the need to directly integrate with all of these concepts, understanding them will help you know what the HL7 engine is actually doing. Envelope: Each communication from one system to another is contained in a block of data. This block of data (Envelope) could contain one or more messages. Message: A message is a set of data that is logically related (for instance, about a specific patient encounter or hospital admission). A message is bounded by its type (Message Type) and contains a number of (Segments). Message Type: A message type is the definition of the structure of a communication and provides rules for what data sections (Segments) are required, which are optional, and the order of the data. The order of the data in a message is important as one section (Segment) can have different meanings depending on where it is in the message. Messages are usually referred to by their message type and event type, for example: an ADT^A01 is an ADT message with an event type of A01 which means that this is an Admission, Discharge, or Transfer type message (ADT) and is signaling the event that a patient was Admitted (A01). Event: Events allow a message type to have slightly different meaning as described above. Events are related to the message type and an important part of the standard. The same message structure can be used to signal different types of events. The same basic data is needed to admit and discharge a patient with only small differences. HL7 has evolved over many years and new events have been added to the standard that weren’t there in previous versions. For instance, an ADT A01 (ADT Patient Admit) and an ADT A08 (ADT Patient Data Update) were initially defined as different message types, but later combined. So, a message that comes in as
an ADT A08 in version 2.5 of HL7 will actually have the structure of an ADT A01, however, because it is sent as an A08 - the meaning of the message will be an update. Confusing? No worries, most interface engines hide this fact and make you think you are still getting an A08.
Segment: A segment is a section of a message that contains the actual values that are being sent between systems. Generally, a segment can be thought of as a line in a message and it always starts with an identifier that indicates what the segment type is. An example of a segment is “PID” (patient identifier) where information about a person or patient is sent, or “MSH” (message header) which is described below. Segment identifiers are always 3 characters. Message Header, MSH Segment: Every message starts with an MSH segment that declares information about the message including its version, type, which text delimiters are used, etc. This special segment is used to provide information as to how to interpret the entire message. Optional, or Z Segments: HL7 is a flexible standard and allows for users to send data between systems that the specification does not accommodate. This is done by using Z segments, which is a custom segment with an identifier that starts with Z. Most people use the Z and then follow it with characters that are part of the specification. For example, to send additional data about a patient identity someone might use ZID, a derivative of PID. Data Type: Every segment is composed of data elements. Each of these data elements has a “type.” Types may be number types like integer and decimal or dates, or more complex types defined by the standard. These data types are positional in the ‘segment line’ and delimited by the defined character (usually a pipe, |) for the message. A data type might have one or more pieces to it (itself being delimited by another delimiter). For instance, a ‘Code’ data type has an ID and Description, but it is one element inside the segment. Delimiters: While the characters that can break up a message are by convention pretty standard, they can be defined in the MSH segment. Each message
B
BASICS segment is always on a new line, and each segment is subdivided into data by a pipe character, ‘|’, and each data type is subdivided into its elements by a caret, ‘^’. See example message below for more information. ACK/NACK: Acknowledge or NOT acknowledge the receipt of a message. Most HL7 systems can be configured to send back receipts of communication. This allows systems to keep a careful audit of data sent between them. Sending systems can generally be configured to resend or at least report messages that get no receipt or get a NACK (not acknowledge message) returned. Interfaces: An interface can mean slightly different things between software and hardware vendors, but essentially an interface is usually a connection between two systems. When an HL7 data feed is configured from the Emergency Department in a hospital to the Electronic Medical Record system this is usually referred to as an interface. Some interfaces are one way - they either send data to another system or listen to data from another system. Most interface engines or interface technologies support sending and receiving data as these technologies normally sit in between two medical systems and modify the messages as they are sent. Different interface engines might have functionality to transform or route messages attached to interfaces. There are two primary technologies used to send message in most healthcare applications: TCP and Files. Direct Connection (TCP): Data can be transmitted over a network connection using TCP (also referred to as LLP or MLLP). This is often referred to as a ‘direct’ connection or a real time connection between applications. File Connection: A simpler way of transmitting messages is by sending the data in a file or multiple files. This can involve loading it into an application or configuring a directory to watch for new files to appear. A file connection can also take the form of an FTP or SFTP server.
B What does an HL7 Message Look like? HL7 messages are made up of segments. Each carrying specific information about anything from a patient’s name to an allergy, a radiology image, a transcript, etc. MSH|^~\&|MegaReg|XYZHospC|SuperOE|XYZImgCtr|20060529090131-0500|| ADT^A01^ADT_ A01|01052901|P|2.5 EVN||200605290901||||200605290900 PID|||56782445^^^UAReg^PI||KLEINSAMPLE^BARRY^Q^JR||19620910|M||20289^^HL70005^RA99113^^XYZ|260 GOODWIN CREST DRIVE^^BIRMINGHAM^AL^35 209^^M~NICKELL’S PICKLES^10000 W 100TH AVE^BIRMINGHAM^AL^35200^^O |||||||0105I30001^^^99DEF^AN PV1||I|W^389^1^UABH^^^^3||||12345^MORGAN^REX^J^^^MD^0010^UAMC^L||678 90^GRAINGER^LUCY^X^^^MD^0010^UAMC^L|MED|||||A0||13579^POTTER^SHER MAN^T^^^MD^0010^UAMC^L|||||||||||||||||||||||||||200605290900 OBX|1|NM|^Body Height||1.80|m^Meter^ISO+|||||F OBX|2|NM|^Body Weight||79|kg^Kilogram^ISO+|||||F AL1|1||^ASPIRIN
In the message above you’ll notice some things that are ‘bold’ to call your attention to them. 1. MSH the message header segment identifies the message. 2. A DT^A01 shows us that we have a patient being admitted (A01) and the message structure is an ADT message. 3. EVN is the event data with some key dates like when the message was sent 4. P ID is the patient identifier. This has medical record numbers, names, and contact information. 5. PV1 is the patient visit data, which details this specific visit to a facility. 6. OBX is an observation, like vitals taken by a nurse 7. AL1 is the description of patient allergies
BASICS What type of data does HL7 Transmit? The HL7 specification is fairly comprehensive. It contains data about many aspects of health care and data including patients, schedules, appointments, interactions between providers and patients, insurance information, information on diagnoses and procedures, medical records and much more. If an application is configured to receive all messages of all types from another application, it is likely that much of the data that is received is not relevant to what is needed. For instance, if I have a scheduling application, it might not be relevant for me to get information on updates of patient allergies - but changes to patient demographic information is very important.
Challenges presented by HL7 HL7 has several challenges that often make it difficult or costly to directly connect two systems.
Multiple Versions The standard has evolved over time, and new capabilities including segment types, message types and event types have been added. Systems that support newer versions of HL7 may have trouble receiving messages from older systems that use an older version of the standard. This can become especially problematic when you have multiple systems that are exchanging the same data and each of those systems is on a different version of the standard. This is further complicated by the fact that people can use Z Segments to evolve the standard according to their needs. If you had an older system to which you added a Z Segment that is accommodated in a proper segment in a newer version of the standard you may need to do a lot of work to transform that data or interface to get data to another system.
Nesting makes it complicated Segments can sometimes be used in more than one place in a message and will have different meaning based on where they are located.
Non-Conformity There are lots of situations that arise in healthcare information technology that cause people to deviate from a standard. For example some manufacturers building monitored equipment might have very limited communication capabilities and might take short cuts to send HL7 (i.e. sending data that isn’t quite valid HL7 format). People with existing data exchanges might make use of a segment type or a piece of data in a segment they know is not currently in use to transport some data between systems rather than creating a Z segment or dealing with a change that might involve a developer. This kind of non-conformity might start off simple or be done pragmatically with the best intentions but over time this makes managing more applications and hardware elements more difficult.
Communication Technically Challenging When connecting health care systems you not only need to consider these details of HL7, but you also have to deal with the technical details of the communication or transport layer. TCP/IP ports, directories on the file system and file permissions all have to be considered
Dirty Data Another problem with HL7 is a problem that’s common to all hospital and computer systems: dirty data. Mis-keyed addresses, or street name formats, system limitations and all kinds of other conditions often cause important pieces of data to be different from system to system. HL7 engines that sit in the middle of your environment and move messages in between systems may have the ability to check and clean that data to increase accuracy and improve care.
Integrations After you get data you have to do something with it. One of the HL7 standard’s goals was to allow systems to share data without having to design a completely custom communication mechanism between those two systems, but rarely is it possible to simply point two HL7 systems to each other and have them successfully share data. This is largely due to the limitations and concerns outlined above. Even if it was simple to simply point two systems at one another, there’s often a need to have customization because of the way your company operates. For example: you may have a custom database of patient addresses that you want to update instead of simply allowing a message to pass from vendor system to another. Integration to custom systems and services in your environment means that your systems and services have to support HL7 natively or you need a “middleware” that can let you perform integrations while messages are passing from system to system.
HL7 Tools: Interface Engines vs. Processing Engines Middleware that sits in between two software systems and helps solve some of the problems outlined above are called HL7 Interface Engines. This last generation of tools focused on helping solve the problems of moving data in and out of systems and databases. This was, and still is, a real challenge and these technologies do a good job in automating much of the plumbing that is needed to have a robust integration between systems. The focus of these integration engines is on getting, parsing and integrating the data into systems. The natural evolution of these engines is to move away from being a place where data is interchanged to a spot where business logic can be enforced. Having business logic running at the touch point between applications allows for straightforward processing like data cleansing and validation but also for more interesting logic like pattern detection, complex routing, or creating tasks for users. These next generation engines - HL7 Processing Engines - contain a ‘superset’ of the functionality of a normal interface engine - and also contain business process, analytics, or business rule functionality.
HL7 Processing Engines The 20 th Century Solution Interface engines and processing engines both need to manage the plumbing of HL7 data structures and communication. Regardless of how processing engines and interface engines differ, these foundational pieces are described below and almost all HL7 engines deal with these elements.
Communication The standard communication methods that are used to send HL7 data follow 2 basic patterns: file based and socket based. While these patterns are reasonably standardized and consistent, since they predate other now ubiquitous communication patterns like SOAP and REST - they need to be built, tested, and deal with areas of the specification that are open to interpretation. Often having existing systems expose a new connection has additional cost to the system vendor as well. Interface engines allow these communication points (interfaces) to be created and have to produce good information to ensure that the data is properly passing over the communication boundaries into the actual application.
Integrity of Message Receipt or Delivery of data Beyond the basic communication protocols, interface engines ensure the transactional responsibility for the receipt and processing of messages. This includes ensuring that messages, once received can be handled. Sometimes this means that the receiver believes it can process the message and in other systems it might mean that the receiver has completely finished processing the message. When a technical error occurs you want your engine to be able to store and queue messages to be processed later.
Scripts to manipulate messages Once a message has been received, manipulation of the message is usually required. Manipulations might include editing the entire message itself (like combining or splitting messages, validating the message structure and message data, or augmenting the message with additional information or segments) or operations on the data. Most interface engines use a scripting concept or language similar to JavaScript that allows a user to write ‘scripts’ that deal with message manipulation and transformation. With some engines there are graphical tools that allow common operations to be done in addition to the scripts. Scripting is really a form of coding where the number and type of operations as well as the language syntax may be less technical than a full featured programming language, but usually script or script-like processing is best done by developers.
Figure 1 - Scripting in Mirth
Figure 2 - Data Mapping in Decisions
Basic system integrations The three common use cases of interface engines are: • Bringing data into an existing software application • Sending data out of an existing software application • Sitting between two software applications that either do not share HL7 compatibility or are not HL7 interfaces. All of the interface engines have ability to do ‘basic’ integration – (ex. deal with basic file/data formats like CSV, Excel, XML, JSON) or do some level of integration with external systems like databases or services. All of them can move beyond ‘basic integration’ by using scripting or programming to do real integrations - however integration that requires aggregation with other data, multiple interactions or deep data transformation is something that uses the integration tool as just a host for programming code. The integration engine is adding little here.
Interface Visibility It is important that the people maintaining and relying on an HL7 interface can view what is happening with an interface. Interface and processing engines both provide some metrics about messages moving into and out of the system, and the availability and connectivity of the systems that are sharing data.
Message viewing HL7 Processing Engines and simpler interface engines needed to provide users with a way to view messages. This may be temporary to diagnose problems, or it may be a part of the day-to-day management of a data feed that needs manual care and feeding. HL7 is usually transmitted in a readable format so it is usually very simple to see the data in a log file or a “data grid� that has a list of messages you can simply copy and paste into a text viewer. There are also third party applications designed specifically for viewing HL7 messages that are very useful, (ex. like 7-Edit - http://www.7edit.com). Some engines have editors like the one shown below which break down the structure of the message and make it easier to see and analyze.
Figure 3- Viewing an HL7 Message in Decisions
Decisions HL7 Processing Engine Decisions for HL7 is based on a full business process automation, workflow, and rule engine. While both interface and processing engines are middleware and must perform the same basic plumbing tasks as described above, they have significant differences that are rooted in a different target audience and a broader set of capabilities beyond the handling of data. Decisions for HL7 can benefit your HL7 implementations in many ways, but the primary benefit of using the Decisions HL7 Processing Engine is that it lets you involve people. It lets you involve the right people in the design and development process, and it lets you involve people in the day-to-day operations in a more efficient way. A processing engine that has workflow in it allows you to involve people by assigning tasks, routing activities to queues, and getting people to provide input at critical points in the process. Two simple examples are keeping patient demographic data clean by having someone review it before it gets committed to your database, or it can be used to alert people of critical lab results or unexpected data.
Subject Matter Experts vs. Data Jockeys Decisions for HL7 is more than a next generation processing engine. It includes a set of visual designers and a visual data mapping capability that allow subject matter experts to be involved. This is a critical advance over engines that require you to write code or even code-like scripts to transform and manipulate data. HL7 deals with data that can directly impact patient care, and being able to involve the right people is critical. Decisions uses a workflow designer that lays out your processes like a flow chart. Every step that needs to be taken with your HL7 message is laid out in a visual process, which makes it possible to sit down and explain it to the stakeholders. Consider this example where a message is received, the patient’s address is checked against a database, the address is corrected, and then the message is sent back out to an EMR by the interface. In Decisions this process looks like this:
Figure 4- A Flow in Decisions
The process is a series of steps, and the steps can do the same things that developers typically do using code. This includes the types of activities people need to do with the structure and data in HL7 messages like: ● ● ● ● ● ● ●
Get Segments Get Segment Data Change Segment Data Add Segments Validate Data in Positions Change a Message’s Version Treat a Message as Different Type
One thing that makes Decisions more powerful than a simple interface engine is that the steps are not limited to HL7 activities. Steps can perform data transformation, external systems integrations (i.e. web services, databases), can connect to social networks, and more. These integrations don’t require complex coding exercises or typical development cycles to accomplish. In Decisions for HL7, even data mapping between different structures is done visually as shown below.
Figure 5- The Data Mapper in Decisions
The biggest benefit to using these visual designers is that you can have the business experts, or medical team involved in designing the process directly. The process diagrams can be understood by all the stakeholders of a process and used as a formal way to agree on the requirements of an interface and document the existing behavior of an interface.
Automation: Rules vs. Data Processing vs. Business Processes Businesses operate the way they do because they have rules, policies, process data, and have workflow processes. Most people think of these as distinct problems that while interconnected are really different classes of problems. It is actually a misconception that a workflow process involving users and forms, status reporting, state, etc. is a very different class of problem than a rule to say that an offer should or should not be acceptable. Rules/policies, processing data and workflows are more similar than they are different. How they are the similar: - The person making decisions about whether or not the data in a process is valid is very likely the same person who makes decisions about how work is routed and what users are involved. In other technologies having a specific rule engine or a specific workflow capability means that this same person has to learn multiple technologies that are in different classes in order to automate business process. In many cases the person with the process knowledge has to rely on someone else to deal with defining part of the process even though these things are closely related in the way they serve the business. - The smallest scale elements (rules and policies) are the foundations to larger scale problems (like data processing and workflow). It’s a ‘policy’ to decide who needs to approve an item. After a workflow is executed and action needs to be taken in an external system, it requires ‘processing’. - Some business rules involve people. While scheduling a new patient with insurance the payer could be determined automatically, but a patient without a payer could trigger a task for someone to manually intervene by calling (Decisions can even automate the phone call to a user to gather information). - All of the categories of automation could benefit from being built in such a way that they can be understood, validated and possibly adjusted - by the people who are responsible for the business. Differences between the types of automation are less striking, they come down to just a few elements:
- Are there known constraints of the problem domain that allow a simpler user interaction to be presented? Rules (aka, decision points or policies) may be able to be stated in a straight/declarative manner while automation of a business process could involve assignments, escalations, recursion on rejection, etc. The business process design experience will have more options and require a higher level of understanding than a policy or rule decision. For example: A rule or policy might be that a gynecological exam cannot be performed on a man, or that an appointment cannot be scheduled in the past. These are data elements that exist in many types of HL7 messages that ensure the data meet basic conditions before being accepted. A process example might be adding a missing facility name to a record by having a person involved in the service manually entering the facility name. - What level of action is being taken by the automation? - Business rules are generally statements of fact about something like the following: 1. This data is not valid. 2. This patient must have a Medical Record Number. 3. This service requires additional approval. - Data processing generally ‘changes’ and ‘integrates’ data - but does not involve people and is not long running. 1. Take a message, clean the address data and store in a database. 2. Process a spreadsheet of appointments from system A and create appointments in the EMR. These kinds of process use rules but instead of making a declaration about the data that it actually does something with it. - Business processes manage a business concept across all or a part of its lifecycle. It uses rules to decide what to do at different points. It processes data (gets data, changes it, integrates it), but also may involve users inside or outside the organization. Business processes may wait for events to happen, handle exceptional cases, and are generally interested in being tracked by its ‘state’.
Most software that does ‘automation’ has a real focus on only one of these elements. They are usually a rule engine, interface engine, or workflow engine - but this constraint is one that is largely self-imposed by the architecture employed. The difference between a rule and a workflow comes down to the size of the pieces that are being coordinated and to a lesser degree the level of constraint that the designer has imposed on the user. Designer constraint sounds negative but it’s actually a good thing. The more constrained a tool is, the easier it is to use. Modern apps have illustrated this perfectly. (Twitter, Instragram, etc). The less constraints - the more flexibility and the more complexity. By providing experiences where users can operate in a framework that has predetermined boundaries, they do not have to consider elements that are outside of these boundaries. Almost all legacy HL7 interface engines deal solely with data processing. They are not a good technology for defining logic (many have a code based logic layer but stop short of being a true process engine), nor are they capable of managing larger scope problems like workflow. The Decisions for HL7 graphical design tools span this entire spectrum without a change in paradigm. Different views that are more constrained are available for running rules, but the same paradigm including fully graphical construction is consistent across all phases of automation. Even ‘visibility’ (dashboards and simple reports) leverages all levels of the automation paradigm. Rules can be used to provide additional meaning in reports; data processing allows extensions to data to be constructed graphically, actions can be built by connecting workflows and processes to the dashboard data.
Managing the information rather than just transmitting it Interface engines take data in, transform it, and pass it on. Their involvement with the data is a ‘touch’ point along the lifecycle of the data. While the Decisions HL7 engine can operate in this manner, it also can manage data over a longer lifecycle. Here is an example: An ADT A03 or ‘discharge’ message comes into the engine from a system. Based on looking up other data and applying rules, the ‘logic’ part of the processing engine can determine that a follow-up workflow needs to take place. Over the course of the following weeks or months steps need to take place like: - Confirming information with the patient - Asking them (maybe repeatedly over time) to schedule a next visit - Assigning a task to a person to call if they do not schedule (Decisions for HL7 can even automate phone calls and text messages through a service) - Looking for an admit message to make sure they actually arrived The combination of rules (intelligence), ability to proactively talk to other systems (integration), and deal with longer running sequences that may involve communication and users (workflow), allows true automation to happen based on this communication.
It’s Smart… Business Logic An ideal place to run business logic is at system boundaries - primarily because every system has a boundary. This is especially true with HL7 where, in healthcare, all systems communicate using pretty much the same paradigm. Instead of having to figure out how to embed rules inside an application, you are capturing interesting events and managing the outcomes and adjustments based on these events. In the Decisions Platform interfaces can be smart well beyond just properly mapping data from one message into another data system. This is because the Decisions Platform has the ability for logical decisions to be constructed, in a way that is understandable and visible to business experts, as its essential building block
Visibility beyond transaction volume All interface engines have reports that let you know how the engine is doing. How long has it been up? How many messages are being sent? How many errors were generated? These reports are essential and give a view into the technical aspects of the interface engine very much in the same way that the gauges in a car report on the health of the mechanics of the cars engine. Interface engines, because they are focused on the data aspects often fail to allow interesting business conclusions to be extracted from the data that is being sent through it. It might tell you how many admission messages are there but cannot easily report on length of stay or tie admission messages to discharge messages. Systems handling the interfacing of medical systems technology get to see much, if not all, the data coming into or out of your environment. This makes it an ideal place to extract interesting analysis and spot anomalies and exceptions that need to be acted on, very much like the argument for why logic is ideal at a system boundary. Rules play a role in getting meaning out of data, and workflows play a role in allowing users or automated processes to be triggered off of relevant data events or elements. While it is interesting to see that there is an item or exception that must be dealt with, it’s much more efficient to be able to select and trigger an action (user or automated) to deal with that data right from the dashboard that reports it. In the Decisions Platform dashboards are graphically constructed and integrated with both rules and with workflows to allow smart and fully interactive dashboards to be created.
Involving People Involving people in a step of a process is one of the most valuable features of a processing engine. In many environments you have to wait until data gets to a system in your world and then correct it. Think about manually correcting demographic data in the EMR that was mis-keyed during appointment scheduling, or creating a new patient record because a medical record number couldn’t be
found. If you put the work of correcting this data at the system boundaries you get better data moving through all of your IT systems and fewer workarounds that require tribal knowledge and lead to mistakes. In other words, if you rely on a process engine you don’t have to rely on someone remembering that they have to temporarily discharge the patient to change their MRN but then must immediately hit F10 to readmit them. The Decisions Platform has a full workflow and form engine that can involve users to initiate transactions, respond to notifications, and deal with exceptions. These forms can be built completely graphically.
Figure 6 - Designing a Form
Figure 7 - End User View of the Form
Using a rule/automation engine and placing rules at system boundaries - this entire pattern can be formalized into technology. If a later result or exception can be traced to the events that initiated them - rules can be defined and written to correct the input data.
Here is an example: If transactions going out to insurance companies are run across a processing or rule engine, non paid claims or ones coming back for clarification can be tied to the sent data. This provides a place to automate to ensure that the items going out in the future do not have the conditions that caused the failure to take place. This pattern is often called continual process improvement
Intercepting Data and Adding Rules Decisions for HL7 has an optional feature called Interceptor Rules that allow you to stop data at the system boundary and create rules from that data. You don’t have to understand or define all of the rules and combinations of rules before implementing the system, because the rules are added to the system over time using the actual data that’s being moved through the system, which allows the rule engine to “learn” your business. Rules are written to determine which data needs to be stopped for manual review (Intercepted). When intercepted, data can be fixed in a custom user interface OR a rule or can be created to automatically fix this type of data and run on the stream of data going forward to fix future occurrences of this condition. For example: if a system is sending in custom procedure codes or incorrect CPT codes, rules can be written to intercept these. When patterns are established (look for this in description field) rules can be created to auto correct these.
www.decisions.com hl7.decisions.com