RTOS Real-Time Performance vs. Ease Of Use

Assessing performance needs of an application
vs. other considerations

by John A. Carbone, VP, Marketing
Express Logic, Inc.
San Diego, CA

Real-time operating system (RTOS) vendors often make claims of “fast real-time performance,” and “rapid real-time response,” to convince developers to use their RTOS for a given application. The implication is that “faster is better,” and sub-microsecond interrupt response and context switch times are offered as compelling evidence.

Given that “real-time performance” is often cited by developers (see Evans Data, 2002, Figure 1) as paramount in their list of criteria for selection of an RTOS, it’s easy to see why OS vendors respond to this demand for performance with measurements designed to show that their RTOS is faster than the competition, and therefore better. Developers are encouraged to choose an RTOS with low overhead or risk missing the performance requirements of their application.

And, obviously, sufficiently low overhead, high interrupt response and fast context switching matter if the real-time application is going to meet its deadlines. However, if COTS RTOS vendors provide performance capable of meeting the needs of most embedded applications, then application developers need to look beyond the real-time performance for other factors that will make an RTOS best for their application needs. To meet all the deadlines—development, time to market, budgetary and real-time performance—it’s time to assess RTOS options from a broader base than just real-time responsiveness. Let’s look at what industry studies suggest as additional criteria.

What performance is needed?

Before evaluating an RTOS based on its claimed interrupt response and context switch times, it’s important to determine what performance the application actually requires. Just what are the response requirements that must be met by the application all or most of the time? How can developers assess the value of a certain performance metric unless they know what performance is required by their application?

The perception exists that “hard real-time” (see sidebar) interrupt response requirements are difficult to achieve, and that the fastest RTOS should be selected to ensure meeting requirements. This leads developers to demand the highest level of RTOS performance, and thereby leads RTOS vendors to spend lots of time making their RTOS respond faster and faster. Indeed, if such responsiveness were a measure of the benefits of an RTOS, then “more is better.”

But, at some point, “more” is not needed. Once response requirements are met, faster response is of little additional value. For example, on a desktop system, it’s generally understood that keyboard response of about 10 milliseconds is sufficient to keep up with human typing. A faster response would provide no additional benefit, since the human doing the typing won’t be able to hit the next character for at least 10 milliseconds.

Figure-2 Interrupt Response and Context Switch Requirements of Various Applications (VDC, 2003)

While not a “hard real-time” requirement, this example illustrates the diminishing value of exceeding requirements. This underscores the importance of understanding the application’s real requirements, so it can be determined whether the RTOS meets them, falls short, or is overkill.

Furthermore, knowing what is sufficient for an application will enable the developer to avoid paying more for an RTOS that offers responsiveness beyond those requirements, if a less costly alternative exists.

Developers speak

Recently, Venture Development Corporation surveyed embedded system developers and asked them the jackpot question: “What interrupt response and context switch requirements does your application need?” The results are rather telling.

Figures 2 and 3 show that most applications demand responses on the order of 10s of microseconds, and many only require 100s of microseconds! That’s not “sub-microsecond” at all. These findings apply across many application disciplines from consumer electronics to industrial automation, from automotive to telecom, and even into military and aerospace. While a few applications were reported to require under 10 microsecond response, the lowest of all was a 2.0-us

Figure-3 Only 30% of real-time applications require response faster than 50 microseconds, and more than half of all real-time applications can be satisfied with response of 100 microseconds (VDC, 2004)

context switch requirement reported by one respondent. Even a 2.0-us interrupt response and context switch time are within the capabilities of most RTOSes, and are hardly sufficient criteria for selecting one RTOS over another.

n light of this interesting data, it’s possible that most, if not all, COTS RTOSes can easily meet the real-time requirements of most embedded applications. This makes faster responsiveness of little value, and means that developers might want to look at other characteristics of RTOSes they’re considering, rather than just focusing on interrupt response and context switch times.

 

What’s really important?

Dr. Jerry Krasner of Embedded Market Forecasters (www.embeddedforecast.com) recently noted from his evaluation of why projects fail that: “49.8% of embedded designs are competed behind schedule.” Notably, Krasner found this to be “an improvement from the last few years in which the number of late projects was between 52% and 55% and the average delay was 3.3 months.”

Perhaps more telling were Krasner’s findings about developer satisfaction with the designs. Krasner observes, “From 2000 through 2003, developers claimed that 33% of their designs were not within 50% of predesign expectations and that over half of embedded designs were not within 30% of predesign expectation!” Krasner’s studies revealed this to be a consistent finding over those three years.”

In April 2004, Gartner Dataquest released findings from a survey to examine what developers liked and didn’t like about their RTOS. To determine levels of satisfaction users had with the RTOS that they are currently using in their projects, survey respondents were asked to rate the importance of a series of attributes that are associated with the RTOS in general.

The attributes credited for delivering satisfaction are listed according to their measured importance:

• Run time royalties
• Features
• Customer Support
• Processors Supported
• Development Tools Supported

Respondents were asked to rate their levels of satisfaction with the RTOS with respect to these attributes. The importance and satisfaction levels were rated on a scale of 1 to 10 with a rating of 1 corresponding to the least important or least satisfaction while 10 stands for most important or most satisfaction.

The overall satisfaction ratings are presented in figure 6 below. As you can see, 61.5% of the users were extremely satisfied with the RTOS while a small minority, 2.4% were very dissatisfied.

	Importance	Satisfaction	Gap
Run Time Royalties	6.7	6.6	-0.1
Features	7.6	7.5	-0.1
Processor Support	7.6	7.6	0.0
Development Tools Support	7.9	7.3	-0.6
Customer Support	7.8	7.1	-0.7
Figure-5 Customer Importance/Satisfaction Scores (Gartner Dataquest, 2004)

Figures 7 and 8 report the user importance and satisfaction ratings Gartner reported corresponding to each of these attributes. The gap or difference between the importance and satisfaction for each feature is indicated as well. In an ideal situation, the importance and satisfaction ratings for each attribute would be identical meaning there would be no gap. The existence of a gap points towards areas of possible improvement.

Gartner research analyst Daya Nadamuni summarized, “As in the past, the trends seen in the gaps are similar. Respondents are most dissatisfied with customer support and development tools support while they are quite happy with processor support.”

Figure-6 Satisfaction Gap in RTOS Use (Gartner Dataquest, 2004)

Based on this overwhelming evidence that the “usability” factors—not the real-time performance characteristics—are paramount to the success of the project, let’s investigate characteristics that would help meet development time and project deadlines.

Clearly, these findings reveal that a faster interrupt response will not help. What would help a great deal, however, would be software that is easier to use and enables faster development. Developers would then get a product built quickly and be able to more likely meet the overall schedule.

Rather than look for an RTOS with 1 microsecond interrupt response and 50 nanosecond context switching, it might be more useful to find one that is easy to learn and use.

Ease of use

“Ease of use” is a term often used to measure the degree to which an RTOS offers good tools, good documentation, good support, a simple API, and availability of full source code.

In general, it’s a characteristic of the product that enables users to be productive with it in a short period of time, helping developers get projects completed on schedule. As opposed to performance, ease of use is more closely related to the critical factors influencing the timely completion of embedded development projects, according to the information from Gartner, as well as that from EMF and VDC.

True ease of use can have very significant benefits for developers and can help to bring products to market more quickly, with lower development cost, resulting in greater profitability. Even though ease of use is inherently subjective, there are some RTOS characteristics that can be evaluated to help assess its ease of use:

1. Simple RTOS system services. Each system service provided by an RTOS is intended to provide the developer with a specific functionality that can be used by the application whenever needed. For example, sending a message. That service might involve hundreds or thousands of lines of code, and might perform a related combination of operations for the application. If the service API is simple, it’s more likely to be easily understood by the developer, and more likely to be easily learned and used. Moreover, it’s more likely to be used correctly, and not introduce opportunity for bugs through misuse due to difficulty of understanding.
    
2. Good documentation. Developers are justifiably cynical with regard to documentation. Perhaps that’s because developers generally hate to write documentation for their own code, so the documentation task is left for others. This can result in pretty, but useless, documents, or technically rich, but poorly written manuals. If documentation is written clearly, contains relevant examples, is technically accurate, and is well organized, it can make a huge difference in ease of use of the underlying product;
    
3. Intuitive naming convention. Many RTOSes use cryptic, complex names for their services that require a reference guide for the developer to recall what the service actually does. An RTOS with descriptive names for its service calls, organized in a separate namespace apart from application function and variable names, will make development easier, and also will assist in code-review by others who didn’t actually write the code;
   

   Eg:
   (Good): rtos_message_send( p1,p2,p3)
   (Bad): xyxConstructBridge_435(p1,p2,p3,p4,p5,p6)
   

4. Availability of well-commented source code. Many commercial RTOSes are available in source code form. This is very helpful for thorough understanding of the operation of the RTOS, and for source debugging of applications as they interact with RTOS services. But just availability of source code is not enough. The code must be well documented, well structured, and modular;
    
5. Responsive technical support. No matter how well-documented and intuitive an RTOS may be, there will always be a need for responsive technical support from time to time. Customer references can provide insight into the quality, and timeliness of support offered by an RTOS vendor, as can testing of support during product evaluation.

Conclusion

In conclusion, beware of investing time in search of real-time responsiveness that is beyond the needs of your application. Once you understand your application’s performance needs, it’s likely that many RTOSes will meet them. Only if you find that your application has unusually demanding response requirements is it worthwhile to sort out the available RTOSes and find the one that best meets those requirements.

For the lucky majority, it’s more beneficial to spend time looking for an RTOS that offers characteristics that make it easy to learn and use, enabling you to have a better chance of completing your project on-time. Make sure to demand an evaluation copy of the RTOS you’re considering, and see for yourself how quickly you can create an application, and what happens when you call for technical support. Greater ease of use just might enable you to complete development and bring a product to market more quickly, more economically, and more profitably.