EnergyIQ Continues to Change and Grow
By Mike Skeffington
With Spring comes many changes. The flowers and trees are blooming, the birds are back, the ski slopes are closed, and the mountain bike trails are open! EnergyIQ continues to change and grow as well, including some big achievements:
- We have released new upgrades to our solutions
- We released a new hosted service for Competitive Intelligence
- More companies chose to partner with EnergyIQ as their solution provider
- We held our 6th annual EnergyIQ Tech Forum in both Houston and Calgary
- And we welcomed petroWEB’s customers to the EnergyIQ family after acquiring petroWEB’s assets
We have a lot of exciting plans for more growth, some of which we presented at our Tech Forum (summaries below). Look for our webinar series, major new releases, and a new web site in the near future. In the meantime, I hope you enjoy the weather before the summer heat sets in!
- The EnergyIQ Tech Forum Recap
- The EnergyIQ Product Stack
- Moving the EnergyIQ Stack Deployment to the Cloud (Atom I, II & III)
- Data Quality Innovation
- Meet the Team: Mark Wiggins
- Meet the Team: Rawaa Al-Saffar
- Where Can You Find Us?
The Tech Forum Recap
In April, EnergyIQ held our 6th annual EnergyIQ Tech Forum. Our continued customer-base growth pushed us to holding the event, for the first time, in two cities during back-to-back weeks. We’d like to recognize Apache and Cenovus for hosting the event in Houston and Calgary, respectively. The majority of our customers were in attendance with a large representation of partners and prospective companies.
The goal of the Tech Forum is to share EnergyIQ’s plans and technical direction for the TDM application suite in an open forum, with the opportunity for attendees to provide feedback and share experiences. Highlights from this year’s Tech Forum included current and future product direction in the following areas:
- IQinsights – Search Technology, GIS Strategy and Native support for Analytics
- IQexchange – Integration Technology & Strategy, Next Gen Data Loaders
- TDM – Architecture and Roadmap, Data Quality Innovation and Cloud Deployment
In the last several years, EnergyIQ has partnered with clients and vendors to develop and provide the most comprehensive set of commercial data loaders in the industry. At the Tech Forum, we presented a new solution, Competitive Intelligence, which delivers consistent and validated datasets and reports for enhanced competitive intelligence by blending multiple vendor sources in a cloud-hosted, managed solution.
Steve Cooper, President and CEO, concluded the event by presenting the EnergyIQ focus, including adoption of emerging technology and architecture to the EnergyIQ roadmap and what you can expect to see delivered in 2019 and beyond. EnergyIQ has been an innovator in Energy Data Management and is positioned to continue to leverage technology and industry experience and remain an innovator and leader in data management solutions.
The EnergyIQ Product Stack
The EnergyIQ product stack has evolved over the life of the company to meet the changing needs of our client base and in response to industry and technology trends.
- The foundational layer is the Trusted Data Manager which contains decades of experience to manage oil and gas data effectively and efficiently. This is where data governance, quality and business rules are applied to ensure that the business receives the best data available and that they know it. TDM is PPDM gold compliant and leverages many of the standards and best practices developed by PPDM over the years including ‘What is a Well?’, Well identification, Well classification, and business rules.
- The integration layer is IQexchange which manages the flow of information into and out of TDM. This layer abstracts the complexity of the underlying data model from calling applications and delivers long-term application interface sustainability through the definition of Data Objects.
- At the top of the stack, IQinsights, the Insights and Analysis layer, is how we present the information to the business through advanced search and visualization technology based upon the Elastic technology stack. This is also how we deliver dashboards and analytics on huge volumes of data quality information in real-time.
A long-term Data Architecture strategy ties the layers together and ensures that we can take advantage of technology advancements and the benefits of cloud-based deployment. We have deployed the EnergyIQ product stack to the cloud for several clients and are moving quickly towards delivering a fully cloud native solution. This will enable us to provide continuous product updates and automatically scale to meet demand within a SaaS model. Looking beyond this, we see EnergyIQ supporting a DaaS (Data as a Service) model where we deliver fully blended and validated data so that clients can spend their time using data rather than managing it.
Recently, EnergyIQ acquired key assets from petroWEB including the EDB and Navigator platforms. EDB is closely aligned with TDM, providing an enterprise data management solution based upon the PPDM data model. Key features within EDB, particularly around log data management, are being integrated with TDM to further enhance the foundation of our stack. Navigator has advanced map-based data query and display capabilities that will be blended with the IQinsights platform to deliver an exceptional user experience. The end result of the integration will be the same 3-tier stack, but with greater functionality, improved workflows, and a killer interface providing easy access to the best data.
Moving the EnergyIQ Stack Deployment to the Cloud (Atom I, II & III)
It has been a busy start to 2019 for EnergyIQ with the development of new features and improvements coming in our forthcoming 2019.1 release. Behind the scenes, we have been hard at work on the deployment of the EnergyIQ stack in the cloud with support for a similar deployment on-premise.
The goal of Atom-1 is to containerize our existing applications. By containerizing our applications and centralizing our configuration across the various parts that compose the stack, this will immediately simplify our existing installation and upgrades. At the completion of the first phase, many of our key components will be containerized and a handful will support High Availability (HA) deployments.
Atom-2 will involve taking our non-HA components and stripping them down to their bare services so that they can be deployed as scalable HA services and applications. At the completion of this second phase, our deployment will not require a Windows server and will support being deployed on Linux VMs or bare metal servers which will help with cost of ownership in the cloud.
While Atom-1 and Atom-2 will both be able to be deployed on-premise, and in the cloud, we are most excited about Atom-3, which will be the realization of the hard work poured into Atom-1 and Atom-2. In Atom-3, we will continue our support for on-premise deployment, but also offer our EnergyIQ Stack in a provider-agnostic cloud deployment of cloud-native microservices and applications with support for resource elasticity. In short, you only pay for the resources you need when you need them. A minimal resource configuration will always run, and the EnergyIQ Stack will have the ability to perform burst data processing by requesting additional cloud resources during periods of high demand to perform parallel processing of tasks.
Our goal with the completion of Atom-3 and this move to a cloud-native platform is to provide a solution that has a low-cost of overall ownership in the cloud, and that can take advantage of cloud resources on demand to provide truly unrivaled performance without hurting your budget by requiring dedicated resources.
Data Quality Innovation
With the initial versions of TDM, data quality rules were written directly against the PPDM data model in the form of 3-part SQL statements:
- Initial population (all good and bad values)
- Quality test (how many bad values)
- Error message to be displayed
The results were displayed in the form of a grid and could be aggregated to display facets of data quality by region.
While this approach was effective there were some drawbacks including:
- Restricted to run only against the PPDM database requiring a knowledge of SQL and the underlying database
- Slow execution when dealing with millions of wells
- Inability to store and display detailed results over time
It was clear that the industry was changing and that the old ways of measuring data quality would not meet the demands of the future. We made a change a couple of years ago, therefore, to develop a Rules Service that would accept a Data Object and return aggregated and detailed quality results to the calling application in the form of a JSON quality object.
The benefits of this approach include:
- A rule is written against the attributes in an object using a query language that allows complex rules to be developed
- Since they work against an object, the rules are independent of the source database (SQL Server, Oracle, NoSQL) and structure (PPDM, other)
- The Rules can be compiled into a Rule Set with individual weightings being applied to each rule.
- When a Rule Set is executed against an object, the results are returned to the calling application with summary results on the overall Rule Set and individual result details. This is a lot of data and the calling application decides what to do with that data
The Rules Service was called from the TDM Loader to make a determination whether a Data Object should be loaded into the data store or rejected depending upon the quality results. These results were stored in MongoDB and referenced during load error analysis.
The innovation step change came, however, when we started to run the Rules Service against data already within the IQinsights Elastic index, and stored the quality results object as a child of the Data Object itself.
By taking this approach, we are able to query and filter the quality results in the same way that we are the technical data. We are also able to generate sophisticated dashboards that allow us to quickly identify problems and put a plan in place to correct them.
We truly believe that this is a game changer in terms of how oil and gas companies measure the quality of huge volumes of information and display that information through dashboards in real time. It also opens the door to more advance quality analysis through the application of machine learning and the ability to set standards for quality across the enterprise.
Meet The Team: Mark Wiggins
Mark Wiggins is an information technology and GIS professional with over 20 years of experience in the oil and gas industry. Primarily focused on design, development, and implementation of custom and 3rd party software solutions, including extensive experience in oil and gas data management practices for upstream.
Prior to EnerqyIQ, Mark was a Solution and Implementation Architect at petroWEB where he was responsible for planning, implementing, and managing PPDM based software solutions. Mark also held the position as the IT Enterprise Applications Manager for a US based E&P company, helping drive technology related business initiatives across the enterprise. Mark started his career as a GIS Analyst for Koch Pipeline Company, implementing data management initiatives and creating best practices in mapping and records management for hydrocarbon pipeline operations.
Mark has Bachelor of Science degrees in Geography and Computer Science from Oklahoma State University and Northwestern Oklahoma State University, respectively, with graduate level work in Geography from the University of Arkansas.
Meet The Team: Rawaa Al-Saffar
Rawaa Al-Saffar started at EnergyIQ’s Houston office at the end of July as a Client Analyst to provide onsite support at one of EnergyIQ’s major clients. She adopted EnergyIQ technologies quickly with a focus on TDM and IQinsights. Two months later, her work was recognized by EnergyIQ managers, and she was promoted and began work as a Solutions Architect. She loves her work in this position and feels it is the perfect fit for her skills.
Rawaa is a Geoscientist who completed her education at the University of Baghdad and Freiberg University in Germany. After that, she worked for Paradigm Geophysical as a Technical Advisor for Geological Modelling, Data Management and Seismic Data Processing. Then, she followed her passion toward problem solving and development to be an Implementation Consultant for various companies in diverse industries.
She is a world traveler and Hackathons enthusiast. She now calls Houston home and has been recognized by NASA, The City of Houston and already twice winner at EnergyIQ for her innovations and problem solving. She likes photography, nature, travelling, stocks, crypto, tech-gadgets and CARS (especially those that are fast)!
Where Can You Find Us?
We’d love to see you in person. EnergyIQ will be attending and sponsoring many coming events. Some of them are listed here. Please come by and say hello. We’d love to see you!
- May 23, 2019: PPDM Calgary Data Management Social Calgary AB
- June 3-5, 2019: NG Oil & Gas Summit Austin TX
- June 4, 2019: PPDM Data Management Luncheon Dallas/Ft. Worth TX
- June 6, 2019: PPDM Data Management Luncheon Tulsa OK
- June 11-13, 2019: Global Petroleum Show Calgary AB
- June 11-12, 2019: Data Driven & Drilling Production Conference Houston TX
- July 9, 2019: PPDM Data Management Luncheon Houston TX
- July 11, 2019: PPDM Data Management Luncheon Midland TX
- July 22-24, 2019: URTeC 2019 Denver CO