Has Recent Work Introduced Errors?

You may have read this analysis of the errors DBDOC found that had been introduced into a client system over four years.

Here is another summary analysis of the important errors introduced into a different system over five years. DBDOC makes this sort of analysis trivial if you have backups of systems before and after work is done. Here are questions that should be asked:

1. How many errors that could hurt the plant are acceptable?
2. Why were these errors not in front of the DCS team for evaluation?
3. How could these errors, all breaking the rules, yield a clean Composer compile?
4. Should Composer have caught them?

Duplicate ADAPT Blocks and Multiple TSTALM Blocks

There are ten errors that result from the failure of Composer to force a cloned ADAPT or TSTALM block to have S2 or S1 set to -1 when the block is cloned.

Two of these errors are shown below. It is absolutely clear that S2 of block 1060 and S1 of block 1066 both should be 1062, not 2450. The value 2450 is residual from the working logic that was cloned into this logic. This is typical of a class of error made again and again by Composer users.

The block index for the prototype MSDRVR FC 129 Block 1062, centre of a wonderful set of beautiful functioning logic. The highlights show the four extra TSTALM block errors, so there are six of each. Not one of the errors was found when the work was done, compiled by Composer or loaded by the user.

Some Function Blocks are Defaulted as Disabled

This error message shows that one lag function was not enabled when it was added. The effect would show up as inexpicable unintended excursions that are supposed to be smoothed out. Clearly, the default should be to be enabled, not disabled.

Specification out of Range

What does this block do?

We could guess. Why should we have to? Why is this not an error in Composer, as the only legal values are 0, 1 or 2?

TSTQ Tests Block With No Quality

The work being analyzed has only two errors where a block is being tested for quality when it does not have quality to test. Only two!

First, note how the blocks with quality, shown by DBDOC as blue lines, lose that attribute when they go through logic blocks. Here, the cost of the one second lag function is the loss of quality for the pressure signal.

So, bad quality on the pressure signal will not be detected. What is it supposed to do? Simply put the FLOW CONTROL STATION into Manual. Oops!

Summary

There are fourteen potentially significant errors documented here, with many more less severe not mentioned. They should not have been made, and they should have been caught.

Analysis: DBDOC Detects Four Years of Introduced Errors

This is a summary analysis of the important errors introduced into a system over four years. DBDOC makes this sort of analysis trivial if you have backups of systems before and after work is done. Here are questions you should ask.

1. How many errors that could hurt the plant are acceptable?
2. Why were these errors not in front of the DCS team for evaluation?
3. How could these errors, all breaking the rules, yield a clean Composer compile?
4. Should Composer have caught them?

Error Browser Presentation

The builds of 2010 and 2014 versions of the Composer project show 49 significant errors that were left in the system after the work was commissioned. DBDOC shows "new" errors highlighted and allows you to focus on them. Old ones are important, too, but good work should not introduce new ones. After all, Composer is two decades old now and ought to be counted on to protect the users.

Some Function Blocks are Defaulted as Disabled

This error message shows that one lag function was not enabled when it was added. The effect would show up as inexplicable unintended excursions that are supposed to be smoothed out. Clearly, the default should be to be enabled, not disabled.

Looking at two of these messages (for Blocks 3678 and 3679), it is absolutely clear that the plant is not protected by rate limiter blocks (FC8) that were inserted to limit the rate of change. The red warning triangle shows the one we clicked on above. The yellow triangle (called an error marker) would also be visible, and you would see when you got to the logic in your analysis or troubleshooting.

TSTQ Tests Block With No Quality

The work being analyzed has thirty-seven errors where a block is being tested for quality when it does not have quality to test. None of that logic does what it is intended to do.

First, the simple error is clear. Testing quality on an output if a FC 80 M/A MFC/P block simply makes no sense. It does not work. The green arrow shows the erroneous test being carries out.

However, as is often the case, finding bad work makes you look at the intent of the logic. What was intended here?

It appears that either U1 ID FAN SUC PRESS BQ or ID MASTER IN AUTO is supposed to replace ID MASTER DEMAND with 100.0. That will not happen!

Summary

There are forty-nine potentially significant errors documented here, with many more less severe not documented. They should not have been made, and they should have been caught.

DBDOC 10.7 -- Download Your DBDOC Update Today!

We are delighted to announce the release of DBDOC 10.7, with a variety of upgrades, bug fixes, and new features.  A few of the major improvements are noted below.  For more information, please see What's New and the Release Notes.

Built-In PDF Files

If your system contains PDF file documentation, you will be happy to learn that PDFs are now first-class citizens in DBDOC, built directly into the M14 like other document types.  They can be viewed right inside of Hyperview, links to blocks and tag references are made automatically, and they can be searched like other documents.

Links to external PDF documentation are still supported, too.

See Built-In PDF Files and More! PDF File Support in DBDOC 10.7 for more information.

Configuration Change History Support

DBDOC 10.6.1 introduced the creation of DMPCFG files for both the .CFG files and the CLD and CAD sheets automatically when you do a DBDOC build.   DBDOC 10.7 introduces automatic archiving of the files from both the CLD / CAD sheets and the compiled or saved CFG files. When you do your DBDOC build, your DMPCFG files will now be archived with the error files. Also, the last build you did before installing 10.7 will be archived. Thus, you will start with a base for comparison of the last build you did with DBDOC 10.6.1 and the first with DBDOC 10.7.

See Configuration Change History Support in DBDOC 10.7 for more information.

Improved S+ and 800xA PG2 Graphics and MicroStation Drawings.

Display of S+ graphics has been significantly improved. Symphony Plus graphics rendering has been greatly improved. Splines, background colors, lines, etc. which previously were not drawn, now appear, and previously handled elements are now drawn more accurately.  The user can now also specify the path to their Symphony Plus image files in Wizard so that Hyperlink can find and use the correct image specified in a graphic. S+ and 800xA live data display sizing has also been improved. MicroStation drawings have improved text display, rendering and remove duplicate search hits.

Improved Error Detection

Processing has been extended to check more blocks for continuous exception reports.  The use of PID and APID blocks with derivative gain combined with inputs from discontinuous input blocks is reported.

See examples of these errors and the problems they cause here.

"Lost Project" Recovery

DBDOC 10.7 can recover "lost" projects. A feature to recover "lost" projects has been added to BuildPlus, allowing project recovery if are inadvertently "lost" due to a change in UAC level on a machine or a user's privilege level.

There's more...

For more information, please see What's New and the Release Notes, or follow this blog, where we will be illustrating new features and DBDOC capabilities regularly.

Click here to get your DBDOC 10.7 Update now!

Configuration Change History Support in DBDOC 10.7

Archived Text File Dumps of Configuration and CLD / CAD Source

DBDOC now reports the specifications that make up the configuration of an INFI 90 system in text files we termed "DMPCFG".  We use these dumps to cross-check DBDOC handling of the CLD and CAD files, verifying that we handle the CLDs and CAD sheets the same as the ABB compilers Composer and WinCAD (and DOS CADEWS and SLAD).

DBDOC 10.6.1 introduced the creation of DMPCFG files for both the .CFG files and the CLD and CAD sheets automatically when you do a DBDOC build.  Read about them here.

DBDOC 10.7 introduces automatic archiving of the files from both the CLD / CAD sheets and the compiled or saved CFG files. When you do your DBDOC build, your DMPCFG files will now be archived with the error files. Also, the last build you did before installing 10.7 will be archived. Thus, you will start with a base for comparison of the last build you did with DBDOC 10.6.1 and the first with DBDOC 10.7.

Changes from Date to Date

The simple application of the DMPCFG file archives is comparison and documentation of changes in blocks and specifications from any DBDOC build dates. The following examples show how any differences will be clear.

This example shows the changes made in a module in the space of three months. Having them available like this will allow them to be cross-checked against the management of change documentation. The symbol "<" indicates the old value, with ">" showing the new blocks and specs.

Comparison of May 8 to Sept 14, 2016 - Module 6,50,05

< Block 531 FC80 S1=3646 S2=5 S3=3493 S4=5 S5=0 S6=5 S7=6.0 S8=0.4 S9=1.0 ...
> Block 531 FC80 S1=3646 S2=5 S3=3493 S4=5 S5=0 S6=5 S7=6.0 S8=5.0 S9=1.0 ...

< Block 796 FC30 S1=5    S2=0  S3=0.0 S4=250.0 S5=300.0 S6=-10.0 S7=0.1
> Block 796 FC30 S1=4706 S2=17 S3=0.0 S4=6.0   S5=10.0  S6=-1.0  S7=1.0

< Block 797 FC30 S1=5    S2=0  S3=0.0 S4=11.0   S5=100.0  S6=-10.0 S7=1.0
> Block 797 FC30 S1=4708 S2=18 S3=0.0 S4=1E+006 S5=1E+006 S6=0.0   S7=1.0

< Block 3546 FC6 S1=5    S2=11.0 S3=-1.0
> Block 3546 FC6 S1=2003 S2=2.0  S3=0.0

< Block 3886 FC37 S1=1    S2=1
> Block 3886 FC37 S1=4714 S2=4719

< Block 4049 FC2 S1=961.0
> Block 4049 FC2 S1=1565

< Block 4050 FC15 S1=4048 S2=4049 S3=1.0 S4=-1.0
> Block 4050 FC15 S1=4710 S2=4056 S3=1.0 S4=-1.0

< Block 4051 FC2 S1=274.4
> Block 4051 FC2 S1=1000

< Block 4052 FC17 S1=4050 S2=4051 S3=1.0
> Block 4052 FC17 S1=4721 S2=7    S3=1.0

< Block 4053 FC2 S1=0.0
> Block 4053 FC2 S1=1.065

< Block 4055 FC9 S1=4052 S2=4120 S3=4054 S4=0.0 S5=0.0
> Block 4055 FC9 S1=5    S2=5    S3=0    S4=0.0 S5=0.0

< Block 4056 FC16 S1=2003 S2=4055 S3=2.0
> Block 4056 FC16 S1=4722 S2=4049 S3=1.0

< Block 4062 FC80 S1=4057 S2=4056 S3=4058 S4=4060 S5=4059 S6=5 ...
> Block 4062 FC80 S1=4057 S2=4713 S3=4058 S4=4060 S5=4059 S6=5 ...

< Block 4075 FC80 S1=4069 S2=5    S3=4070 ... S28=3061 S29=0    S30=0 S31=60.0
> Block 4075 FC80 S1=4069 S2=4053 S3=4070 ... S28=3061 S29=4716 S30=0 S31=60.0

< Block 4120 FC68 S1=0 S2=1.26 S3=0.74 S4=0.84 S5=0 S6=5
> Block 4120 FC68 S1=0 S2=21.5 S3=12.5 S4=12.5 S5=0 S6=5

< Block 4581 FC16 S1=4580 S2=6 S3=1.0
> Block 4581 FC16 S1=4580 S2=6 S3=0.05

< Block 4582 FC6 S1=4581 S2=200.0 S3=-5.0
> Block 4582 FC6 S1=4581 S2=10.2  S3=4.2

New blocks added:

> Block 4706 FC7 S1=4707 S2=0.115
> Block 4707 FC6 S1=1063 S2=2800 S3=0.0
> Block 4708 FC166 S1=796 S2=2 S3=5 S4=1 S5=9.223E+018 S6=-9.223E+018 S7=1.0 S8=1
> Block 4710 FC17 S1=4056 S2=4052 S3=0.5
> Block 4711 FC17 S1=4050 S2=4051 S3=1.0
> Block 4712 FC15 S1=5 S2=5 S3=1.0 S4=-1.0
> Block 4713 FC30 S1=4711 S2=117 S3=0.0 S4=5.0 S5=9999 S6=-9999 S7=0.5
> Block 4714 FC12 S1=4076 S2=1.07 S3=-9999
> Block 4716 FC35 S1=3886 S2=0 S3=2.0
> Block 4717 FC2 S1=21.5
> Block 4718 FC2 S1=12.5
> Block 4719 FC62 S1=0 S2=1 S3=0 S4=0 S5=0 S6=4719 S7=0 S8=2
> Block 4720 FC9 S1=4717 S2=4718 S3=4719 S4=0.0 S5=0.0
> Block 4721 FC30 S1=4720 S2=43 S3=10.0 S4=25.0 S5=-9999 S6=9999 S7=0.5
> Block 4722 FC3 S1=3546 S2=1 S3=0.0 S4=2.0

What Next?

This is a respectable first step, but of course these are text files, generated during a DBDOC build, and not nearly as accessible as they could be.  We would like to see support for interactive viewing and browsing of Management of Change information from right inside the Hyperview browser.

Andrew McKeown of Oji Fibre Solutions in New Zealand suggested:

• My biggest need is around Management of Change. Knowing what has changed between each snap-shot build is an important part of managing and reconciling or even tracing changes between successive builds or across a period of time.  My contention was always that if you have the information about what is there then we should do something about seeing what has changed in detail.
• My contention now is that you could build this into the M14 by changing the file contents so that the differences could be seen in the MoC list [showing] the change and also the hyperlink to the CAD sheet and Block.

As far as comparing from version to version arbitrarily, Andrew also sets the bar very high:

• The trick then becomes what do you choose as the previous version to compare to [with] CFG and EWS CFG as sources plus the previous versions of the same.
• Archiving files for comparisons to previous versions for an ad-hoc datetime difference is all well and good but you need to make this interface easy for the user.

We concur, and are starting on this and more now. The ideas of our clients will help us.  There was a time when the errors DBDOC found during builds were available only via build-level log files, but now the Hyperview Error Browser makes this information highly accessible, and links it effortlessly into graphics and configuration.  There isn't any reason why Management of Change information, changed specs and so on, couldn't be presented in Hyperview in an analogous fashion.

Would this be useful to you?

Text File Dumps of Configuration and Source -- A Useful New Tool

Since Version 10.6.1, DBDOC has reported the specifications that make up the configuration of an INFI 90 system in "DMPCFG" text files. We use these files to cross-check DBDOC handling of the CLD and CAD files, verifying that we are handling the CLDs and CAD sheets the same as the ABB compilers Composer and WinCAD (and DOS CADEWS and SLAD).

An offshoot of this work was that we were able to make DBDOC generate the same values as ABB compilers in the cases where we found the CFG files do not have the documented default values. Two examples are S28 of FC80 and S2 of FC 225. The documentation says each has a default value 2, but Composer and WinCAD compile a value of 0.

At the moment, you need to compare and interpret these text files manually, but look for DDBOC to do more automatically in the future.

The following example occurred this past week, and was something of a mystery to track down. DMPCFG proved its value as part of the solution.

Example 1: Mismatch Between CLD/CAD Files and the Module Load File.

A client was concerned at the failure of the blocks shown here (8120, 9498) to display live data in Hyperview.  As you can see, other blocks on the same sheet were showing live data without incident.  They speculated that the cause might have something to do with the fact that one of the blocks was an AOLDB, a new block type. 

DBDOC Analysis:

In fact, the block type issue was a red herring.  DBDOC does in fact support the newest block types added to Composer.   The fact that the values of the specifications were shown correctly was evidence for that.

The actual problem was almost certainly that the configuration in the module did not match the CLD, and the CLD is of course what is used to generated DBDOC's graphical representation of the CLD diagram.  We expected that most likely, for some reason, blocks 8120 and 9498 did not exist in the module, but that they were present in the CLD.

In order to verify this, we had a look at the DMPCFG files from the CFG (loaded into the module), and the CLD (used by DBDOC).  Surprisingly, they showed the exact opposite of what we were expecting!

CFG DMPCFG File -- 8120 and 9498 are present!

Block 8120 FC48 S1=9498 S2=16 S3=0.0 S4=50.0 S5=20.4 S6=-99.0 S7=1.0 S8=2.0 S9=2.0  

Block 9498 FC96 S1=5 S2=1254 S3=9497 S4=9.223E+018 S5=9.223E+018 S6=9.223E+018

And in the CLD DMPCFG file, 8120 and 9498 were absent. 

So then, how did 8120 and 9498 get into the CLD diagram displayed in Hyperview?  And why weren't they in the module at the time when live data was being requested and the screenshot taken?

DMPCFG File Comparison

It's clear from the contents of the DMPCFG files provided by the client, that at the point the DMPCFG files were created, the CFG did have blocks 8120 and 9498, while the CLD did not.

Left: CFG (with block 8120), Right: Source CLD (without 8120) 

Left: CFG (with block 9498), Right: Source CLD (without 9498, and with new blocks 9499 and 9501) 

What Actually Happened

This is what the sequence of events turned out to be:

1. By October, a CLD with block 8120 FC 48 and block 9498 FC 96 had been compiled into a CFG that was loaded into the module.
2. This same CLD was built by DBDOC into the October M14 file shown, where blocks 8120 and 9498 are present.
3. The CLD was changed to have blocks 8120 and 9498 removed, and blocks 9499 (a new DAANG block) and 9501 added.
4. A new M14 file was built in November, which was not being used when the problem was noticed.  However, it was this DBDOC build that generated the DMPCFG files in the comparison above, since at that moment, the CLD no longer had blocks 8120 and 9498. It had not yet been recompiled, so the CFG file remained unchanged as shown blocks 8120 and 9498.
5. Later, the CLD without blocks 8120 and 9498 was compiled, and the resulting CFG loaded into the module.
6. Finally, the October M14 was accidentally viewed in Hyperview, instead of the newest November one, leading to the CLD "NO CONN" screenshot first noted, as Hyperview attempted to fetch live data for blocks that no longer existed in the module.

Of course, one might ask, what caused the client to accidentally view the second most recent DBDOC M14 file, instead of the most recent one?

It turns out, when Hyperview opens an M14 specified with '*' (most users do this, because it's supposed to allow the most recent build to be opened automatically), it checks the File Creation date, which can be changed to the current date by copying the file.  This is what happened with the M14s in question -- they were copied, and the less recent of them received a newer File Creation timestamp, and thus was erroneously opened by Hyperview as the most recent M14.

We are changing Hyperview to instead look at the File Modified time for this purpose, and this modification will be seen in DBDOC 10.7.  Hopefully this will protect clients from this problem in the future.

Built-In PDF Files and More! PDF File Support in DBDOC 10.7

You all have PDF files, integral to the support of many systems. DBDOC now supports them in an unprecedented way.

DBDOC has supported viewing PDF files with a PDF viewer for many years. This has been very useful. However, this support lacked the full suite of features that are integral to DBDOC. The new support adds these critical features:

• You can search for text in all the PDF documents you build in at once. Conventional searching is limited to the document you have open.
• You can have links from the PDF file to tags in your DCS configuration. This means you can simply double-click on the tag name link and go to its source on a CLD or CAD sheet.
• You can also have links from the PDF file to other documents. Since DBDOC supports AutoCAD, Microstation, PDF files as well as your DCS configuration and graphics files, this can give you powerful inter-connections in your documention.

Let's look at the three ways that PDF files are now supported in DBDOC.

1. The new support to compile them into the M14
2. The existing support to link external files to be viewed by a PDF viewer
3. The existing support to the Function Code PDF file documentation from ABB

1. Compiling PDF files into the M14

You can now view and browse pdf files directly in Hyperview.  For most people, this support will simply replace the linking they used to do. DBDOC will be even more useful than it was. Searching, tag  and document links will be automatic. You will find it natural and satisfying.

2. Linking external PDF files to be viewed by a PDF viewer

So why would you link PDF files? One client has a very solid application involving maintenance orders. By linking fixed file names to the most recent version of the document, people get to the current version. When it is updated, the new version is linked, because the name has not changed.

This feature remains supported, so you can have very good access to the information in PDF files by compiling them, while supporting access to system-wide documents by name.

3. Supporting the Function Code PDF file documentation from ABB

DBDOC implements support for calling up the ABB Function Code PDF file information by a "right-click" on the function block image. The feature works for individual files, plus for one-volume and two-volume document sets.

Where do we go from here?

We are at the starting point in our development of this support. Your feedback about things we ought to think about will be very useful. The support is important to DCS systems supported by DBDOC.

A System Without DBDOC - Analysis II

This analysis focuses on the 219 errors in this site where TSTQ blocks test blocks inappropriately. There are 209 inputs to TSTQ blocks being tested that do not have quality to test. In addition, quality is being tested on ten AO/L blocks that do not have quality on their input signal.  I began this analysis in the earlier article: A System Without DBDOC - Analysis I and continue it here. Keep in mind that this is a running power plant!

TSTQ Testing Blocks Without Quality

To keep this brief, the images will show the block being tested. Be confident that it is feeding a TSTQ block input and that the intended protection for the plant is simply not present. Also, you might not be familiar with the feature of DBDOC that shows lines that can carry quality in blue, to contrast with lines that cannot carry quality in black.

Engineering Errors

It turns out that, in this system, somebody did not get the point about the TSTQ block. Half of the errors show places where the logic state of a function block is [1] in the alarm condition, but that state is fed into a TSTQ block instead of affecting the process by being input to an OR operation directly. 

TSTALM Block 8272 output does not have quality (note that it is a black line).  Therefore the TSTQ Block 8917 that tests it will never trigger due to this signal.  Nor will it trigger due to the actual situation the system designer presumably wanted to detect: that of TSTALM Block 8272 being [1].

The two instances of testing the low output (H//L Block 9763) are just as incorrect. The result should be OR'd, not tested for quality.

In these and similar situations, operational issues will fail to be detected as intended. There are 105 instances where a TSTQ tests a TSTALM [FC69] block, which does not have quality.  There two instances where a TSTQ Block tests a H//L [FC12]  block N+1, which also does not have quality

Blocks Not Propagating Quality

A common misconception is that AO/L [FC 30] and DO/L [FC 45] blocks are guaranteed to show bad quality. In fact, they only propagate the quality attribute. Thus, if the input to the block does not have the quality attribute, there will be no quality to test.

There are seven instances where a TSTQ tests an AO/L block, which is not propagating quality, and three instances where a TSTQ tests a DO/L block, which is not propagating quality.

A Sublime Example

Here is a sublime example of a class of error that occurs again and again. Good engineering is clutched from the jaws of victory! Here, TSTQ and REDAI blocks validly synthesize a picture of two versions of COMPENSATED GAS FLOW. However, the use of Transfer [FC9] Block 1835 destroys the quality chain, so bad quality will not be detected by the process. The problem would not have existed if a REDAI block had been used instead to propagate this signal.

More Flawed Quality Tests

In all the following instances, quality is checked to no end, and the actual quality information is lost.  In most cases, simply testing the input to the function block whose output is being tested would solve the problem and allow a bad quality signal to be detected and acted upon.

• 2 instances - TSTQ tests F(x) [FC1] which does not have quality
• 2 instances - TSTQ tests A [FC2], which does not have quality
• 2 instances - TSTQ tests H/L LIM [FC6], which does not have quality
• 2 instances - TSTQ tests SQRT [FC7], which does not have quality
• 20 instances - TSTQ tests T [FC9], which does not have quality
• 8 instances - TSTQ tests SUM(K) [FC15], which does not have quality
• 23 instances - TSTQ tests [NOT FC33], which does not have quality
• 3 instances - TSTQ tests S R [FC34], which does not have quality
• 33 instances - TSTQ tests OR (2-Input) [FC39], which does not have quality
• 1 instance   - TSTQ tests OR (4-Input) [FC40], which does not have quality
• 1 instance   - TSTQ tests REMSET [FC68], which does not have quality
• 4 instances - TSTQ tests M/A MFC/P [FC80], which does not have quality
• 1 instance   - TSTQ tests TRIG [FC171], which does not have quality

Conclusion

This analysis show a plant operating with hundreds of errors DBDOC has detected by flagging TSTQ block inputs that do not have quality to test. Half of them are especially worrisome because they show incorrect engineering that was never detected, i.e. it was the original intention to test a signal, not the quality of the signal, in the first place.  Was there a FAT (Factory Acceptance Test)? If so, it missed them all.

A Support Query: Importing the Wrong Value from an M/A Station Block

Dear Support:

I was wondering if you could explain this error to me.

Dear Client:

For sure.

One place block 3088 is used is as a block imported by an AI/L.

The exception report from an AO/L block looks like this:

• one byte - status bits, including H, L, Q
• one real - value of the AO/L

The exception report from a STATION block looks like this:

• one byte - status bits, including H, L, Q, Deviation
• three real values in this order
•     value of the PV (S1)
•     value of the SP (output N+1)
•     value of the CO (output N)

The AI/L block reads and supports the first two values - one byte, from which it gets the quality, and one real, which it imports as the value.

The message tells you that the value of the AI/L block imported equals that of the PV of the station block, something like 239 (S1), not the CO value of 46% (Block 3088), which is probably what is expected.

Please send the same screen shot, but also the AI/L block.




Dear Support:

The AI/L block

Dear Client:

As you see, the description PLANT AIR FLOW CONTROL OUTPUT just does not cut it. In fact, 46% would not be not high enough to trigger the 50% level in the subsequent H//L block, but the erroneously imported PV of 239 is.

You could insert an AO/L after the station block, to transmit the actual output value to the AI/L block.

However, a simpler way might be to put an H//L block on the page where the Station block 3088 exists and send the output of the H value to a DO/L, then import that DO/L here.

Does this now make sense?

A System Without DBDOC - Analysis I

This is the first of a series summarizing the severe errors found by DBDOC at INFI 90 sites that do not have our product for system integrity protection. Be assured that there are more errors in this system than the ones shown here, but this article highlights some of the glaring ones. The system has 5723 configuration sheets at a large power plant. This site gets no warning from its existing tools about the errors described below!  DBDOC, however, would be able to alert it to all of them.

Adapt Block Cloning Errors

There are two adapt block cloning errors, caused by failure to change S2 to match cloned logic. Two subsystems are not adapted, even though they look like they are.

[S2 of block 6115 should be 6116, not 5116]        [S2 of block 7814 should be 7815, not 1115]

TSTALM Cloning Errors

There are nine multiple TSTALM situations. Every one of these comes from S1 not being changed when logic was cloned. Fourteen actions can happen mysteriously when they should not, triggered by the prototype logic instead of the new logic. Even worse, fourteen actions do not happen when they should, because the cloned logic is not being tested.

[S1 of block 3130 should be 3122, not 2920]

Why fourteen? One instance was cloned badly six times.

TSTQ Block Inputs Without Quality to Test

There are 209 inputs to TSTQ blocks being tested that do not have quality to test. In addition, quality is being tested on 10 AO/L blocks that do not have quality on their input signal. Three of these flawed tests control the manual interlock of control loops, so the loop will not go into Manual when the quality is bad.

[Analog Transfer block 4307 FC 9 does not have quality to test]

It is disconcerting that a number of the errors flagged look like OR and TSTALM block outputs are being handled incorrectly by being inputs to TSTQ blocks, instead of being OR'd with the outputs.

REDAI Blocks Used Without Being Tested for Quality

There are 46 instances where REDAI blocks can synthesize bad quality, but the value is then used whether or not it is bad. The first one I checked can be bad quality but still control a "FD AIR MASTER" control loop.

The one shown here is kind of cute, if you like scary productions. Good engineering was undone.

[Analog Transfer block 1835 FC 9 does not have quality to impart to the AO/L block and tag]

Function Block Errors

Among the function generator blocks, there are

• four that change from slope 1:3 to 1:1 instead of clamping
• four that change from slope 4:1 to 1:1 instead of clamping
• two that change from slope 7:1 to 1:1 instead of clamping
• three that with slope 1:1 that also do not clamp

[The large equal value (X,Y) pairs mean that the output rises at the same slope as the input]

Too Few Inputs
One 8 input Qualified OR block has four inputs, but needs seven to trigger it. It ain't gonna happen!

[Qualified OR block 7273 FC 36 will be a 1 if S9 (value 7) of the 4 inputs are value "1"]

Rung Block Errors

Two rung blocks ignore an input without documentation. If a test is in progress, too bad!

[S3 value 20 of FC 111 block 5026 means that value of "TTY TEST IN PROGRESS" is wiped out]

One rung block tests vibration limits, but does not disable the result on bad quality, as designed.

[S6 value 0 of FC 111 block 2606 means that value of  the test quality block is ignored]

Other Errors

There are 29 DAANG and AO/L blocks giving continuous exception reports.

There are 15 DAANG block tags that are designed to show some or all of 2nd and 3rd high and low alarms that will not do that.

One MA Station has type 5, not defined. Another has its initial mode after startup value set to 0, also not defined.

Summary

339 errors are outlined above. This is 339 errors more than the site knows about. What risks do these errors represent to the site? Why do these errors exist undetected? Should this system even be running?

Output Spikes - Sharp as Tacks!

This does not look good!

Here is a warning about derivative gain from a site, highlighted in the Error Browser.

PID Module 1,06,02 Block 214 uses derivative with PV from discontinuous input Module 1,06,02 Block 140

You can see the error marker next to the PID block it concerns.

Dan Reynolds sent me the data from the control loop identified. Here is what it looks like with DBDOC WatchWindow at 250 ms.

Zooming in a bit, we see that Block 280, the output, suffers spikes from its railed value of -5.0%. Block 214 shows a value of +1.7408%, that is, 6.7408 above the -5.0% it should be. The exception reported input Block 140 changes each minute and drives these spikes (not all of which are picked up in the 250ms reporting interval, since they are so brief). You can be sure that, if the control output were in the 0 % to 100% range, large positive and negative spikes would be suffered every minute. Ouch!

Just as clearly, if this PID output were not driven to -5 %, the spikes would be bi-directional and still significant. The effect is purely caused by the value of Kd on S8 for Block 214, only 0.2, but enough to drive the output in a way that is not valid.

By the way, you should notice how any exception report input has steps in its value. The values for Module 1,06,02 Block 140 and Module 1,03,04 Block 1901 show these characteristic steps. See Monitoring Exception Report Performance for details.

Alarm Management Needs Error Management

Does your system contain alarms that cannot actually ever occur?  Many do.

Alarm management focuses on understanding and controlling the alarms that can be generated in a system.  DBDOC provides techniques for walking through the configuration systematically to validate assumptions and look for problems. Effective alarm management is impossible without error management and configuration analysis.

Here are three example sets of alarms that do not work in a single powerplant configuration:

1. Intermittent failure caused by duplicated exception report imports
2. Detected failures to test blocks with quality
3. Problems that can be found by using DBDOC to walk through the configuration easily

If you do not manage your errors and use the tools DBDOC provides to audit the configuration, you will not be able to manage alarms. Even worse, you can spend a great deal of money fooling yourself about alarms.

Example 1 - Frozen Import Block Error

INFI 90 allows block values from one module to generate "exception reports" when the block value changes status or value. In the case of an analog exception report, the new value is transmitted when it changes by an amount that is specified as "significant".

If a an exception report block value is imported more than once into a module, only one of the import blocks works. The values of any other blocks importing the exception report block value will remain frozen. Furthermore, when an on-line configuration is done, a second import block will silently begin working (!) and the block that was working will freeze at its last value. This applies to both analog and digital exception reports.

DBDOC detects and reports this error. However, when the module is compiled using Composer, the error is not detected.

      Error: Frozen import block [093]

      Duplicate off-module reference (DI/L, DI/L) to Module 1,01,05 Block 1230: 1020324 and
      1020342 [1010536]

The source value above is imported in two places in one module. This instance on a spare block page might have the active connection.

The following shows the intended logic.

There is a 50/50 chance that the correct logic is not working at any time. Simply put, tags 0D0730 "COND RETURN TANK HIGH/LOW LEVEL" and 0D0727 "AUX DEA HIGH/LOW LEVEL" might alarm, but they might not. It is a crap-shoot!

Note: It would seem reasonable to assume that AI/L block 1675, now spare, actually started life feeding S3 of block 1676. The duplicate import situation was discovered and the logic changed properly to get both values of S3 from a single AI/L block somewhere in the past. The problem was that the spare block still can be the active one.

Example 2 - Test Quality of a block that does not have quality to test

Powerplant A has 7707 TSTQ blocks. These four input blocks give a binary [1] as a result if one or more of their inputs have bad quality. Actually, 8592 blocks are being tested in all. Of these, 316 do not have quality to test at all.

Here is one example of an alarm (and shutdown) that will not work, identified by the message here.

       Error: TSTQ tests block with no quality [063]

 
       TSTQ Module 4,05,03 Block 2273 tests Module 4,05,03 Block 6170 (FC 15), which does not
       have quality [4050378]

Obviously, the simple solution to the problem of no alarm on bad quality for 1 GROSS MW TRANSDUCER A is simply to test the quality of the input at IREF P1E0388A instead of the output of the fudge factor sum block 6170.

Example 3 - Misconfiguration of a DAANG Block

DBDOC also allows you to go looking for trouble. Walking through all instances of a Function Code will allow you to look for things that were missed long ago. This example shows bad quality that will not alarm.

DAANG blocks are powerful tools when configured competently. Powerplant A has 6227 of them, the second highest number we have seen in a single plant. I was curious to see how consistently programmed they were. The table shows how a value places on Specification S8 can set bad quality.

This plant uses this extensively. This example shows how the constant 32.0 is applied to S8 with the green highlights. When the raw signal is found by the TSTQ block to be bad quality, the DAANG block will be marked bad quality as intended.

The following example shows that undetected errors exist. The red highlights show that bad quality on this raw input will result in good quality being forced. Oops!

Being interested, I started walking through the 6227 DAANG blocks looking for this particular issue.  I got bored after 200, taking 16 minutes, about 5 seconds per block. I found (and flagged with DBDOC annotations) the following reality:

• 124 blocks with correct alarm configuration - setting 32.0 into S8
• 22 blocks incorrectly setting 1.0 into S8
• 1 block incorrectly setting 0.0 into S8
• 53 of the first 200 DAANG blocks did not set the quality attribute in this way.

The bottom line is that 12% of the first 200 DAANG blocks (all tagged, by the way) would not alarm on bad quality. I have no reason to think this is not representative, which suggests that perhaps 800 of these tags will not alarm on bad quality in this one system.

By contrast, I checked a system with 7980 DAANG blocks. There were no alarm errors in the first 150 instances, with all quality inputs being generated correctly. At least in this respect, good configuration work would make alarm analysis valid.

In a second system with 5084 DAANG blocks, I found that only 2042 of them used the S8 to set bad quality if needed. In fact, this simple walkthrough with DBDOC showed that that 60% of the blocks I sampled were fed from signals that had no quality to test, but did not reinsert bad quality. That could mean about 2000 simple errors - tag does not show bad quality when it should.

Error and Configuration Management Trump Alarm Management

Alarms cannot be managed if errors prevent them from working. DBDOC can be used to improve alarms both by looking at errors and by walking through the project carefully. Otherwise, any alarm analysis will be incomplete and unacceptable.

DBDOC: Reads the Manual and Checks Your Specs.

DBDOC is unique in giving you significant help in finding integrity issues in your system. Every INFI 90 system in the world would benefit from improving operational safety and reliability. Our Error Browser and Error Marker development was designed to make it possible to manage errors and improve the integrity of your systems.

Here you see our much loved copy of the function code manual, dog-eared and duct-taped. You've probably read it too, but possibly not as intensely as we have.

Based on studying the documentation and the experiences of our clients, we added new tests for function block specifications that violate the documentation in ways that seem significant. Read on to see what we found.

We reviewed test data for 229 systems around the world, and found 1160 new errors of note in 77 systems involving 33 function codes - 15 errors per system. Of course, this means that 68% of the systems had none of the specification errors we looked for. It also means that over 1/3 of the systems had these new errors Composer did not find, but that DBDOC will.

I expect this will be the most boring blog I ever write. Holy Understatements, Batman!

Here are example of errors found in various specification values for various function codes from real systems. Some could bite!  All are now reported by DBDOC in the Error Browser.

FC35 - 45 instances in 21 systems

• Module 5,65,02 Block 133 FC35 S2 value 005 is out of the valid range [000,102]

FC36 - 15 instances in 4 systems

• Module 5,26,05 Block 6441 FC36 S9 value 9 is out of the valid range [0,8]
• Module 7,101,06 Block 5800 FC36 S10 value -1 is out of the valid range [0,1]

FC45 - 22 instances in 6 systems

• Module 2,48,02 Block 854 FC45 S2 value 20 is out of the valid range [0,2]

FC50 - 1 instance in 1 system

• Module 3,01,04 Block 4082 FC50 S1 value -1 is out of the valid range [0,1]

FC69 - 9 instances in 4 systems

• Module 32,04,05 Block 3285 FC69 S2 value 5 is out of the valid range [0,2]

FC80 - 61 instances in 26 systems

• Module 2,78,02 Block 2151 FC80 S6 value 39 is out of the valid range [1-3 or 5-8]
• Module 2,15,05 Block 2039 FC80 S17 value 246 is out of the valid range [0,7]
• Module 1,11,02 Block 2494 FC80 S23 value 5 is out of the valid range [0,4]

FC82 - 4 instances in 3 systems

• Module 3,02,20 Block 1992 FC82 S5 value 30 is out of the valid range [0,2]
• Module 2,12,06 Block 15 FC82 S15 value 179 is out of the valid range [0,1]

FC83 - 501 instances in 4 systems

• Module 11,30,15 Block 463 FC83 S1 value 65 is out of the valid range [0,63]

FC84 - 5 instances in 4 systems

• Module 2,20,02 Block 6641 FC84 S2 value 10 is out of the valid range [0,1]
• Module 2,20,02 Block 6641 FC84 S2 value 10 is out of the valid range [0,1]

FC86 - 1 instance in 1 system

• Module 2,104,10 Block 1882 FC86 S6 value 013 is out of the valid range [000,101]

FC95 - 36 instances in 5 systems

• Module 1,62,03 Block 979 FC95 S7 value 2 is out of the valid range [0,1]
• Module 1,53,02 Block 9062 FC95 S12 value 7 is out of the valid range [0,1]

FC110, FC111, FC112 - 24 instances in 10 systems

• Module 5,12,04 Block 4015 FC110 S1 value 20 is out of the valid range [0,3]
• Module 5,23,02 Block 56 FC111 S1 value 10 is out of the valid range [0,3]
• Module 7,09,03 Block 4140 FC112 S1 value 10 is out of the valid range [0,3]

FC123 - 12 instances in 1 system

• Module 4,07,06 Block 3793 FC123 S7 value 22 is out of the valid range [0,11]
• Module 4,07,06 Block 3793 FC123 S8 value 22 is out of the valid range [0,11]

FC124 - 6 instances in 2 systems

• Module 5,33,03 Block 3194 FC124 S11 value 120 is out of the valid range [00,22]
• Module 5,33,03 Block 3194 FC124 S13 value 60 is out of the valid range [00,22]

FC126 - 3 instances in 1 system

• Module 1,14,11 Block 8501 FC126 S2 value 115 is out of the valid range [0,2]

FC129 - 92 instances in 33 systems

• Module 1,15,02 Block 3405 FC129 S7 value 003 is out of the valid range [000,111]
• Module 1,15,06 Block 3133 FC129 S8 value 1000 is out of the valid range [000,111]
• Module 1,20,05 Block 7810 FC129 S9 value 002 is out of the valid range [000,111]
• Module 1,23,03 Block 3995 FC129 S10 value 1000 is out of the valid range [000,111]
• Module 5,15,04 Block 2559 FC129 S11 value 20002 is out of the valid range [0000,2222]
• Module 1,44,04 Block 2509 FC129 S13 value 0003 is out of the valid range [0000,2222]
• Module 1,24,03 Block 1808 FC129 S14 value 222 is out of the valid range [000,142]
• Module 1,04,03 Block 3993 FC129 S15 value 23 is out of the valid range [0,1]
• Module 3,54,04 Block 7764 FC129 S19 value 5 is out of the valid range [Any or All of 0,1,2,3]

FC132 - 64 instances in 14 systems

• Module 1,01,15 Block 1346 FC132 S3 value 2 is out of the valid range [0,1]
• Module 1,60,02 Block 448 FC132 S3 value 4 is out of the valid range [0,1]
• Module 11,26,04 Block 1106 FC132 S3 value 7 is out of the valid range [0,1]
• Module 34,46,03 Block 8150 FC132 S10 value 78 is out of the valid range [0,5]
• Module 1,03,04 Block 1211 FC132 S13 value 50 is out of the valid range [0,5]
• Module 2,01,02 Block 244 FC132 S16 value 150 is out of the valid range [0,5]

FC136 - 28 instances in 3 systems

• Module 1,07,03 Block 1258 FC136 S16 value 3.0 is out of the valid range [0.0, 1.0, 2.0]

FC140 - 34 instances in 4 systems

• Module 10,02,07 Block 1076 FC140 S4 value 8078 is out of the valid range [00,11]
• Module 7,103,02 Block 3250 FC140 S5 value 212 is out of the valid range [0,63]

FC143 - 1 instance in 1 system

• Module 1,20,09 Block 8872 FC143 S1 value 3 is out of the valid range [0,2]

FC149 - 60 instances in 10 systems

• Module 5,41,02 Block 232 FC149 S3 value 8 is out of the valid range [0,1]
• Module 11,01,04 Block 161 FC149 S11 value 5 is out of the valid range [0,2]
• Module 11,01,04 Block 161 FC149 S12 value 5 is out of the valid range [0,2]
• Module 11,01,04 Block 161 FC149 S13 value 5 is out of the valid range [0,2]
• Module 11,01,04 Block 161 FC149 S14 value 5 is out of the valid range [0,2]
• Module 11,01,04 Block 161 FC149 S15 value 5 is out of the valid range [0,2]
• Module 11,01,04 Block 161 FC149 S16 value 5 is out of the valid range [0,2]
• Module 11,01,04 Block 161 FC149 S17 value 5 is out of the valid range [0,2]

FC151 - 6 instances in 3 systems

• Module 1,65,05 Block 4531 FC151 S6 value 4532.0 is out of the valid range [0.0,127.0]
• Module 2,30,03 Block 7180 FC151 S7 value 3 is out of the valid range [0,1]

FC156 - 5 instances in 4 systems
Module 1,18,03 Block 6438 FC156 S19 value 32 is out of the valid range [0,1]

• Module 3,03,02 Block 7284 FC156 S20 value 40 is out of the valid range [0,1]
• FC166 - 8 instances in 3 systems

Module 4,05,07 Block 3697 FC166 S2 value 120 is out of the valid range [0,2]

• Module 3,01,04 Block 3807 FC166 S8 value 32767 is out of the valid range [0,1]
• FC182 - 19 instances in 1 system
• Module 1,07,14 Block 56 FC182 S1 value 000 is out of the valid range [001-009, 020-124, 040-041, 064]

FC190 - 6 instances in 3 systems

• Module 2,32,07 Block 2280 FC190 S2 value 7200 is out of the valid range [1,6553]

FC216 - 61 instances in 10 systems

• Module 1,09,07 Block 8172 FC216 S3 value 19 is out of the valid range [1,16]
• Module 20,38,07 Block 650 FC216 S4 value 051 is out of the valid range [000-109, 010-113, 020-125, 040-144, 060-161, 099-199]
• Module 1,03,04 Block 1428 FC216 S5 value 2 is out of the valid range [0,1]
• Module 6,60,08 Block 162 FC216 S11 value 0 is out of the valid range [16,24]

FC222 - 1 instance in 1 system

• Module 1,09,02 Block 1023 FC222 S2 value 1022 is out of the valid range [0000-1209, 0010-1115, 0210-1214, 0300-1302, 0400-1400, 0500-1500, 0900-1900]

FC224 - 18 instances in 4 systems

• Module 1,09,02 Block 1048 FC224 S2 value 1047 is out of the valid range [0,2]
• Module 1,09,02 Block 1048 FC224 S3 value 1046 is out of the valid range [0,255]

FC226 - 8 instances in 2 systems

• Module 20,10,31 Block 8358 FC226 S3 value 10 is out of the valid range [0,4]

FC247 - 4 instances in 2 systems

• Module 1,13,04 Block 953 FC247 S5 value 5 is out of the valid range [1-4 or 6-8]

Holy Complications, Batman! A number of expressions in http://holysmokesbatman.com/directory apply. DBDOC helps improve your system's integrity.

Output Cusps - Exceptionally Artistic!

We came across the pretty but alarming output forms shown here at a client. The scale is one minute per cusp and just over 1% amplitude. This is important because it means that the HMI and historian would not see this at all. DBDOC Watch Window handles it nicely, as is evident.

So what is going on here?

The blue line is the output of a FC 80 Station block and the output cusps were, in fact, a significant issue. The green line is the raw PV input (nice and smooth). However, the APID block was being fed by an exception-reported version of the PV, shown in red. The significant change specification was 0.1%, so the exception-reported value actually was competently implemented, changing in steps, as expected.

The problem simply is that the APID algorithm used a derivative term. As the two diagrams make painfully obvious, this simply does not work. Although the PV signal is changing at a nice fairly constant rate, the controller, fed by the red step input, is hit with a huge change accumulated over one minute, but seen as happening in 250 ms. Thus, the derivative is calculated as being 240 times bigger than it actually is.



The second image shows that, if the PV is changing so that the exception reports are triggered by the significant change, the cusps are smaller, as they have a better time base. but they are still misleading and problematic.

The bottom line is that PID and APID algorithms should never use a derivative term if the PV is affected by an exception-reported input PV. It cannot work mathematically, and it sure kicks the process in the teeth, again and again.

DBDOC can't yet detect this error situation automatically, but we are working on it.  Stay tuned.