At the heart of all of our solution implementations is an industry leading Feed Processing Engine (FPE). Our technology allows us to not only quickly adapt to different sources, format changes and new feeds, but to evaluate business rules and correlate reference data in memory, before the records are even inserted in a database. This is the key to the quick deployment of new sources, less customer reliance on the vendor and faster data analysis.
The Feed Processing Engine (FPE) is highly scalable to accommodate hundreds of parallel feeds. The system utilizes a shared memory architecture, which allows multiple components to process data in separate threads while being mediated into the database. This significantly enhances the performance of the system while allowing a system to be sized to the subscriber base. This same architecture allows the system to scale elegantly to accommodate growth either organic, or from the acquisition of other carriers.
Some of the important features of this platform are:
- Volume Handling – Capability to process billions of transactions each day
- Flexibility – Rapid adaptation to different sources, format changes and new feeds
- Data Enhancement – In-memory correlation with other sources, filtering and business rule application prior to insertion into the database
- Linear Scalability – The architecture of the Feed processing engine accommodates as-you-grow the addition of hardware to provide a linearly scalable platform
- Data Cleansing and Validation – Source data is automatically validated for format and data type consistency
- Comprehensive Error Handling and Logging – Errors are logged to allow for 100% data accountability
- No Data Loss – The Feed processing engine includes buffering capabilities to prevent any data loss regardless of frequency and size of the data source as well as mechanisms for reprocessing and outage handling
- Stream Monitoring – Alerting when expected volumes and processing speed rates are not met per source while statistics are computed for trending and analysis
- Common Format Output – After processing transaction files in various formats, the Feed processing engine outputs the records in a common format for upstream processing