50% of Systems Have Certain Function Blocks Unintentionally Disabled. Is Yours One of Them?

DBDOC is unique in giving you significant help in finding integrity issues in your system. Every INFI 90 system in the world will benefit from improving safety, reliability and operation with DBDOC's Integrity Plus approach. Our Error Browser and Error Marker development was designed to make it possible to manage errors and improve the integrity of your systems. 

 

In our most recent versions, we have added tests for three function blocks that are disabled by default.

• FC 3 - Lead / Lag - S2 defaults to 0 disabling the function
• FC 8 - Rate Limiter - S2 defaults to 0 disabling the function
• FC 166 - Integrator - S4 defaults to 0 disabling the function

For a function block to be disabled by default is horrific. Only three of the 200 types or so are (the rest are enabled by default). The hapless users put function blocks in place, but they do nothing that they were intended to do, because they are disabled. Now and forever, they are booby-trapped.

What do you think? Can there be any justification for most function blocks being enabled by default, with only these three exceptions?

Am I wrong? Are there other booby-traps we have missed? Please let us know.

In my review of test data for 228 systems around the world, I found

• Lead / Lag - 605 disabled in 83 systems
• Rate Limiter - 392 disabled in 86 systems
• Integrator - 50 disabled in 26 systems

In actuality, 75 systems had one of these; 42 systems had two and 8 hit the TriFecta! In total 125 of the 228 systems had one to three of these errors - 55% of them.

What about the "real world" - places where DBDOC checking has not been used yet? Here is what this test detected in a small power plant being worked on now by extremely competent people:

• Module 1,02,02 Block 8507 FC8 disabled by spec values
• Module 1,02,02 Block 9541 FC8 disabled by spec values
• Module 1,02,02 Block 10295 FC8 disabled by spec values
• Module 1,02,02 Block 11049 FC8 disabled by spec values
• Module 1,02,02 Block 11803 FC8 disabled by spec values

 

Holy Rate Unlimiter, Batman! http://holysmokesbatman.com/directory contains 350 actual sayings, perhaps the best being "Holy Time Bomb".

 

The 5 errors out of 152 Rate Limiter blocks show that only DBDOC can protect you from these problems.

 

Here are examples of each from real systems.

 

The objective of this FC 3 Lead/Lag block is to apply a time constant of 60 seconds to changes in the PV as applied to the APID control algorithm. This is not happening here because, unbeknownst to (or unnoticed by) the DCS specialist, the block is disabled and feeds the input directly to the output.

 

The first Rate Limiter example looked like it was booby-trapped in the sense that sometime in the future an unsuspecting body would put meaningful values on the limits and expect them to work. As you can see here, the negative going rate limit of 0.00333 per second presumably is intended to prevent larger drops in the value from affecting the process. However, the block does not function.

 

 

 

In the case of the integrator, it is S4 that is the trap. Clearly, nobody puts an integrator and tag into a system not intending to get data. This one will not get data because S4 is not wired to a constant value 1.

 

Only DBDOC can improve your system integrity by detecting these relatively common oversights.

 



FC 222 and 223 Can Have Severe Exception Report Problems

Summary

If you have FC 222 and FC 223 blocks, you should check for the following problems:

1. FC 222 S8 defaulted to 0.0 and FC 223 S9 defaulted to 0.0 which causes continuous exception reports if the block is tagged in any HMI.
2. Spare FC 222 and FC 223 blocks tagged but not used which cause absolutely wasted continous exception reports.
3. FC 222 and FC 223 blocks with all the significant change specifications set to 1, even when this value is not appropriate.

Details

At a client site, DBDOC was responsible for uncovering a significant loading problem involving FC 222 - Analog In/Channel that has been causing significant loading of the node communication capability. It turns out that FC 223 - Analog Out/Channel has the same problem. Here is an outline of the problem:

• site has significant number of FC 222 S8 and FC 223 S9 blocks at default value of 0 significant change
• even worse, many of these have "spare" tags identifying unused blocks brought into the HMI by exception report
• the result is exception reports generated by the node at once per second, many utterly wasted, and the rest far too sensitive to be valid.
• examination of other systems shows that some have all default values replaced, although there is clearly no understanding that the value is in engineering units (like FC 177) not in percent

DBDOC tools including the Significant Change Report and the extraction of all specifications made it easy to look for the problems once they were conceptualized.

In the site in question, a power plant installation, the bulk of the FC 222 blocks were done relatively recently, taking advantage of modern INFI 90 hardware.

There are 573 FC 222 blocks with default significant change of 0. Our tests with DBDOC Watch Window easily proved that each of these blocks, either if imported or in the HMI tag database, generated an exception report every Tmin seconds, that is, every second. In fact, 92 more FC 222 blocks had non-default significant change, because they had been properly configured.

What about these 665 FC 222 and FC 223 blocks?

• One had no tag, but was imported by an AI/L block, so it is generating an exception report anyhow.
• 196 were named spare blocks, none used in graphics or PI, so196 XR's per second wasted.
• The other 469 blocks were giving an XR every second.

The PCU distribution of the load was:

• PCU 2 - 640
• PCU 3 - 16
• PCU 4 - 8
• PCU 5 - 1

There actually were 2181 tags defined in PCU 2, which means that every Tmax of 60 seconds, each generated an XR. Thus, the base XR load was:

• 573 per second from default FC 222 blocks
• (2181 - 573) / 60 = 27 per second from the other tags

That is, the base XR load on the node is 600 XRs per second. It is a good thing this system has NIS 21 / NIS 22 communication cards. As it is, this showed that the average communication CPU usage in the node of about 50% was a valid figure.

Why worry about this?

554 of the 640 tags and export blocks had default significant change. If the defaulted FC 222 blocks had 1% significant change, the base load would nominally be 2181 / 60 sec or 36 XRs / sec. Allocating 600 XRs per second nominally would allow the analog tags to be increased in sensitivity by a factor of 20, with less load on the system than there is now.

This investigation also brought our attention to FC 223, the Analog Out/Channel block, which also has a default significant change of 0.

Guess what? There are 80 of these in PCU 2. They all have the default 0 significant change, so they are generating 1 exception report per second. 31 of them are spare blocks, so that load is doing nothing.

Bottom line is that, unbeknownst to our client, the exception report loading on the node included:

• 96 analog in and 31 analog out spares giving 127 utterly wasted XRs per second
• 469 analog in and 49 analog out tagged blocks giving 518 very, very, precise values every second.
• 26 XRs per second from the other 1536 "low class" tags, most at default significant change.

Thus, the load was about 600 XRs per second. If the spare tags were deleted and the defaulted ones set to 1% significant change, the load would be 518 / 60 per second (typically) or 9.

This node would generate 35 XRs per second as currently tuned. There is lots of XR capability to improve its tags and history data using the capacity that is nearly completely wasted right now.

How many other nodes with FC 222 and FC 223 blocks have this bad situation?

The good:

• Large Australian power plant has 10663. One has default significant change. It is also not tagged.
• Large Canadian power plant has 1610. None of the 161 that have 0 significant change is tagged.
• Large American power plant has 1312, none of which is defaulted.
• Medium size Canadian power plant has 192, all with sig change set to 1.

The bad:

• Large Australian process plant has 66 with default values. 16 are tagged, which is a meaningful load.

The ugly:

• These small American power plants with all values defaulted at 0 and over one-third tagged but spare.
• Small American power plant with 88 of 754 with default 0. 86 of these have tags, so they are a load. 21 are spares, so they are a wasted load.
• Medium size Canadian process plant has 496, all at default 0. 216 are in PCU 171 and 280 in PCU 172. Every one is generating one XR per second.
• Medium size American process plant has 559 FC 222, of which 368 are tagged with significant change 0. No spare ones are tagged.

It is clear that the significant change specifications and tags for all FC 222 and FC 223 blocks should be examined. The load can be large, and it can be a waste of a precious resource, even stalling or losing exception reports.

Postscript

The general situation with FC 222 and FC 223 blocks that are not defaulted is to have the significant change specification set to 1, no matter what the span.  This is probably an error, too, suggesting that the work was done under the misconception that the specification is a percentage value, rather than an EU one.  1 in a span of 5 is 20%, whereas 1 in a span of 1000 is 0.1%.  Since both appeared in the same system, with the only value used being 1, it is likely the values should be studied even when they are all not defaulted.

Function Code Quantity

A client wrote:
     "Can DBDOC tell me how many Function Code 156s we have in our project?"

The answer is:
     For sure!  Let us count the ways. 

1.  In DBDOC Hyperview, under Miscellaneous Indices, click Function Codes and then Function Code 156. This will give you a list of all of the blocks in your system that are function code 156. The blocks are numbered, so you can just scroll to the end of the list for a count.

2.  FC156.dbf in the Exports subfolder has all the FC156 blocks and all the specs, even the compiled input block numbers (and FCnnn.dbf has the same thing for the FC nnn blocks).

3.  File FCLIST.TXT in the build folder lists all the function codes used in the project and how many of each that there are.

4.  The MHD Module Info section of a module will tell you how many are in that module.  This can make module CPU loading possible to estimate.

5.  MASTER.DB is an SQLite database that has all the Function Code information and a lot more. SQLite tools will allow you to open the file, examine the records and do queries.

The bottom line is that DBDOC has a great deal of capability built in that most of our users never find, so they do not tap the resource they have available.  We have tried to do things that are not possible easily with existing tools, if they are possible at all.

Happily, one more site is now turned on to the capability.  

Over the last half year, I have presented Advanced DBDOC workshops and Hyperview training sessions totalling 23 days at 13 sites.  Without exception, the DCS teams found a mess of useful things in DBDOC that they had not guessed were provided.