Ensuring a Quality IEEE 2030.5 Stack Integration

One of the most important parts of the overall project of building an IEEE 2030.5 interface is the development or acquisition of the core protocol stack. Considerations must be made regarding resources, time, and technology available. But regardless of how you acquire your core protocol stack, you need a robust way to test and integrate the stack. That is where the expertise and software solutions of QualityLogic comes in. For a refresher on tackling the overall process of creating an IEEE 2030.5 interface, you can take a look back at this blog. For this one, we will dive into how to test and integrate your developed or acquired core protocol stack.

Testing an IEEE 2030.5 Implementation for CA Rule 21The IEEE 2030.5 message exchange protocol requirements for CA Rule 21 CSIP can be developed by your product development team using the IEEE 2030.5-2018 Specification and Schema, the Common Smart Inverter Profile (CSIP), the SunSpec Test Specification, and QualityLogic’s Client or Server Development Tools.

Your team may also choose to look at third-party vendors to acquire an IEEE 2030.5 stack. This choice can reduce time-to-market. The challenges from a testing perspective, though, are ensuring that the licensed code meets your requirements and that the code integrated into your system remains conformant and functioning. Whether you choose to license a 2030.5 code base or develop your own stack, you can use QualityLogic’s IEEE 2030.5 Test Tools to confidently test the resulting product.

Using the QualityLogic test tools to develop your own code provides the following benefits:

• Windows based software test tool that acts as an IEEE 2030.5 Client or Server to test your device under development against (e.g. we simulate a server implementation to test your client against if you are developing a client or vice versa if you are developing a server)
• Our FTS Test Tools allow users to select which tests to employ to automatically test specific functionality of the implementation and assess the conformance of your device or system to the IEEE 2030.5-2018 standard
• If you are developing more than the CA Rule 21 CSIP implementation, our Ad Hoc testers allow users to understand and test a range of IEEE 2030.5 applications including DER, DR, EV, Pricing, etc.
• All of our IEEE 2030.5 test tools check conformance rules in the message exchanges
• Our real-time messaging logs enable detailed inspection and analysis of the message traffic to diagnose issues
• Test tools are quickly updated to incorporate additions and enhancements to the IEEE 2030.5 standard and test specifications such as the SunSpec CSIP Test Specification.
• Technical support is available
• By acquiring both Client and Server testers, customers can review exchanges between the two golden reference units to understand how specific IEEE 2030.5 functions should work. This is a huge benefit to developers working with IEEE 2030.5 for the first time.

By understanding what the functionality is supposed to look like and by getting to iteratively test your developing code against a “golden reference” implementation, you can have confidence your code is on its way to being a quality product as well as certifiable.

Integrating an IEEE 2030.5 Stack into Your Product
Most likely you will be integrating your IEEE 2030.5 client, aggregator or server code into an existing product, whether it is a communications adapter or the communications module in a cloud-based product or device firmware. In any case, the communications module is simply the software that sends and receives correct messages (i.e. IEEE 2030.5 messages).

The IEEE 2030.5 messages are the start of a chain of events that is intended to result in some sort of real-world behavior of DER systems. For example, a facility DER controller will receive an IEEE 2030.5 message to change programming on a smart inverter or return a message to a server with the status of the DERs under its control. The integration means your software or firmware translates the IEEE 2030.5 message into information that can be used internally to develop control messages for the DERs being managed, then further creates correct response messages in an IEEE 2030.5 message to respond properly back to the server.

Even if you start with a properly functioning IEEE 2030.5 client implementation, the challenges of ensuring the rest of your system properly interprets the messages and properly acts upon them are significant and require as much testing as the message exchange development itself – probably more.

These challenges come in several shapes and sizes:

• How do you simulate correct IEEE 2030.5 server behavior and messages to your client implementation (or vice versa if you are building a server)?
• How can you ensure the messages are being interpreted correctly given the integration and UI the communications code is integrated into?
• How do you ensure the bug fixes and enhancements that will occur during development and product maintenance have been implemented correctly so the product continues to produce consistent and correct results?
• How do you keep your testing up-to-date on any changes to the IEEE 2030.5 technical specification and certification testing requirements?
• If the product must be certified by an independent test lab, how do you ensure the initial and revised versions of your product meet and continue to meet certification requirements?

Development teams tackle these challenges every day and continually look for tools and processes that improve quality while reducing costs and time-to-market. For proprietary communications protocols, the testing must be developed and maintained internally for each protocol supported.

However, when the communications protocol is an industry standard that is used by numerous vendors, the potential for reducing costs and time and improving quality goes up exponentially as there are likely to be mature 3rd party test tools available that can be leveraged.

In the case of IEEE 2030.5, QualityLogic has been developing and updating its’ test suite since 2010. Our tools can save tens of thousands of dollars in test tool development costs while ensuring conformance to the protocol standard. To duplicate these tools would require hundreds of thousands of dollars in investment and ongoing maintenance. Additionally, our tools are used for the SunSpec CSIP IEEE 2030.5 certification testing conducted by a global network of authorized test labs.

QualityLogic’s test tools address the integration challenges for an IEEE 2030.5 implementation by:

1. Simulating correct client or server behavior and assessing the correctness of the system being tested (in terms of its messaging responses);
2. Providing correct test messages that can be used to evaluate whether a conversion to another protocol or provision of information behaves correctly (the tools validate correct messaging, but actual system or hardware correct functioning is evaluated using other tools).
3. Using the test messages from the QualityLogic tools, the round-trip information or command exchange can be evaluated in terms of the IEEE 2030.5 message that is returned to the test system – i.e., a message to change smart inverter curves or controls should result in a response that the commands have been completed. The responses result from conversions to and from other messaging protocols. Further checks can be done by requesting a list of current controls and curves to compare to the selected ones after the commands have been implemented.
4. The tools can be integrated into your regression test system through our recently released API as a robust set of tests you would otherwise need to develop yourself. Test logs of a nightly regression test can be compared to the prior night’s logs to identify anomalies to further investigate.
5. The QualityLogic test tools are regularly updated to reflect changes in the IEEE 2030.5 specification itself or any conformance test specifications must be met. This ensures the testing of updates based on these changes can be done quickly and correctly.
6. Since our test tools are used by test labs for certification testing, using the tools as part of your development process ensures a quick certification and re-certification of your product.

So as you can see, the IEEE 2030.5 Test Tools developed by QualityLogic represent the best way to pre-certify an IEEE 2030.5 implementation regardless of how the core stack was acquired. Under our model, each vendor gets the flexibility to decide how best to use their resources to get the base code they need, and once that is in place, we have a robust testing tool to ensure that all vendors can have confidence and avoid multiple certification attempts.

Beyond CA Rule 21 CSIP
The QualityLogic test tools are capable of more than CA Rule 21 CSIP testing. The new CALSSA Testing Pathway requires a “simulated utility head-end server”. Inverter vendors, aggregators, and test labs are using our IEEE 2030.5 Ad Hoc Client Tester to generate the 5 required test cases. And we recently added a CALSSA Testing Pathway feature to our Client FTS product. Download our Application Guide for more information.

As requirements for the Phase 3 Advanced Smart Inverter Functions start seeing deadlines in CA, our tools are being used to test specific Phase 3 functions – e.g., Functions 2 and 3 at this point. The IEEE 2030.5-2018 Specification and QualityLogic’s IEEE 2030.5 test tools support all of the Phase 3 functions from a messaging perspective. Inverter vendors and test labs are already using our Ad Hoc tester to conduct the Phase 3, Functions 2 and 3 certification testing. Download the Phase 3 Application Guide for more information.

Finally, the upcoming approval and implementation of an updated IEEE 1547.1 testing specification for smart inverter functions includes an Interop requirement. That requirement can be met through either SunSpec Modbus, DNP3 or IEEE 2030.5 communications protocols. The testing requires the messaging protocol to modify settings on the inverters and then the test labs validate that the inverters actually changed behaviors correctly. Our IEEE 2030.5 Ad Hoc tester can support this testing for inverter vendors with some sort of IEEE 2030.5 interface or gateway. We are currently researching additional test support for the updated IEEE 1547.1 requirements.

Interested learning more? Let’s talk.