Packet Serial Parser

Serial console parsing framework provide ability to detect certain logs appearance on the console and report it back to the test. Example of such logs may be “Kernel panic”, “Something crashed”, “U-Boot bla-bla”. The test (if exists at the moment of the event) will be notified ASAP and can take appropriate actions. Examples of actions may include diagnostics gathering, some smart logging, DUT reboot and not touching the DUT. Also it is possible just save serial console output into the TE logs.

How to launch

To use the framework of serial console parsing make sure that the two following steps done.

Build options

To compile the agents with support of serial console parsing framework should be added option –with-serialparse to agent type options:

TE_TA_TYPE([linux], [], [unix], [--with-serialparse], [], [], [], [])

Configurator objects

The TE has auxiliary configuration file cs.conf.serial with configurator objects for the serial parser management. The file should be included to the config:

<xi:include href="cs.conf.serial" parse="xml" xmlns:xi="http://www.w3.org/2003/XInclude"/>

To use event handlers the instance /local: should be added and value of the instance /local:/tester:/enable: should be set to 1. The last to launch the thread on the Tester to handle events.

<?xml version="1.0"?>
<history>
    <add>
      <instance oid="/local:"/>
    </add>
    <set>
      <instance oid="/local:/tester:/enable:" value="1"/>
    </set>
</history>

Simple use cases

There are two ways to configure and manage the serial console parsers: using the configurator and using the Test API. See the examples below for the both methods.

Logging of the serial console

By default the logging is enabled for all active parsers. So to launch the logging should be added and enabled a parser instance. Level of the logs by default is WARN.

Configurator

To logging serial console messages to the main log, insert the following code to the configuration file. The code launches a new thread on the agent Agt_A. The thread establishes connection with a serial console named console_name by port 3105 and gathers messages from the console. my_parser is name of the parser.

<add>
  <instance oid="/agent:Agt_A/parser:my_parser" value="console_name"/>
</add>
<set>
  <instance oid="/agent:Agt_A/parser:my_parser/port:" value="3105"/>
  <instance oid="/agent:Agt_A/parser:my_parser/enable:" value="1"/>
</set>

User can enable, disable or set of message level value of the logging:

<set>
  <instance oid="/agent:Agt_A/parser:my_parser/logging:/level:" value="RING"/>
  <instance oid="/agent:Agt_A/parser:my_parser/logging:" value="1"/>
</set>

There is the variable logging sets to 1, it is to enable logging (0 - disable). The the log level sets to RING.

Test API

The second way to launch logging of the serial console is a using of the Test API. See the following code example to launch the same parser as before, but from a test :

#include "tapi_serial_parse.h"

tapi_parser_id     *pars;

pars = tapi_serial_id_init("Agt_A", "console_name", "my_parser");

tapi_serial_parser_add(pars);
tapi_serial_logging_enable(pars, "WARN");

tapi_serial_parser_del(pars);
tapi_serial_id_cleanup(pars);

By default port for connection to the conserver is 3109. User can change any connection configurations in the parser id structure. For more detailed description see the Test API section.

RCF

RCF API also can be used to launch logging thread. In this case the thread will be launched without any participation of Configurator. See the following example.

<?xml version="1.0"?>
<rcf>
<ta name="Agt_A" type="linux" fake="no" synch_time="no" rebootable="no"
    rcflib="rcfunix">
    <conf name="host">bilbo</conf>
    <conf name="port">18001</conf>
    <conf name="sudo"/>
    <thread name="serial_console_log" when="rcfuser">
        <arg value="rcfuser"/>
        <arg value="WARN"/>
        <arg value="100"/>
        <arg value="3105:user:console_name"/>
    </thread>
</ta>
</rcf>

Event handling

To handle any events, one or more event and handler instances should be added on the Tester subtree. For detailed description of handlers see the Tester section. The value of the handler instance is a path to the executable (binary or shell command). To detect events to agent should be added parser instance. The parser should contain event instance. Use a name of one of the events declared in the Tester subtree as value of the agent event instance, in the examples it is my_event value. Also should be added pattern instances. Patterns used to detect events. An event happens if a serial log message matches at least one of the patterns, associated with the event. Write a pattern to the value of the pattern instance.

Configurator

There is example how to add a parser with events via configuration file:

<?xml version="1.0"?>
<history>
    <add>
      <instance oid="/local:"/>
      <instance oid="/local:/tester:/event:my_event"/>
      <instance oid="/local:/tester:/event:my_event/handler:external_handler"
                value="/home/andrey/work/trunk/handlers/new_handler"/>
      <instance oid="/local:/tester:/event:my_event/handler:internal_handler"/>

      <instance oid="/agent:Agt_A/parser:my_parser" value="console_name"/>
      <instance oid="/agent:Agt_A/parser:my_parser/event:ag_ev" value="my_event"/>
      <instance oid="/agent:Agt_A/parser:my_parser/event:ag_ev/pattern:1" value="fooo"/>
      <instance oid="/agent:Agt_A/parser:my_parser/event:ag_ev/pattern:2" value="bar"/>
    </add>
    <set>
      <instance oid="/local:/tester:/enable:" value="1"/>
      <instance oid="/local:/tester:/event:my_event/handler:internal_handler/internal:" value="1"/>
      <instance oid="/local:/tester:/event:my_event/handler:internal_handler/signal:" value="SIGINT"/>
      <instance oid="/agent:Agt_A/parser:my_parser/port:" value="3105"/>
      <instance oid="/agent:Agt_A/parser:my_parser/enable:" value="1"/>
    </set>
</history>

The instance name of pattern object is index number, should be used only integer numbers greater than zero.

Test API

The second way to handle events is using the Test API. See the following code example to launch the same parser as before, but from a test:

#include "tapi_serial_parse.h"

...

tapi_parser_id      *pars;

...

pars = tapi_serial_id_init("Agt_A", "console_name", "my_parser");
pars->port = 3105;

tapi_serial_tester_event_add("my_event");
tapi_serial_handler_ext_add("my_event", "external_handler", 0,
                          "/home/andrey/work/trunk/handlers/new_handler");
tapi_serial_handler_int_add("my_event", "internal_handler", 0, SIGINT);

tapi_serial_parser_add(pars);
tapi_serial_parser_event_add(pars, "ag_ev", "my_event");
tapi_serial_parser_pattern_add(pars, "ag_ev", 1, "fooo");
tapi_serial_parser_pattern_add(pars, "ag_ev", 2, "bar");

tapi_serial_parser_del(pars);
tapi_serial_tester_event_del("my_event");
tapi_serial_id_cleanup(pars);

Architecture of the framework

Agent

To listen serial console, perform the logging of the received data, to parse data for events a new thread starts on the agent. The serial console parser thread is responsible for logs gathering. The Configurator performs configure and launch it. It listens for the data from the conserver running on the host and receives arbitrary amount of data. The thread parses a data and searches one of events. When event is detected, the thread changes the event state in the appropriate instance. Optional, the thread can to print data from the serial console in the main log.

Tester

The Tester has a separate thread too. It should periodically poll status of the parsers events from the Configurator, when an event occurs the thread calls an appropriate handler or a series of handlers.

Handlers

When an event has been detect, the Tester calls a sequence of handlers. A handler can be internal or external, this indicates the /local/tester/event/handler/internal instance. The handler subtree has priority field, that is used to control of the handlers sequence. The higher priority - the sooner the handler performed.

If a handler is internal and an event occurred, then the Tester should send a signal specified in the handler subtree. The event will be processed by test itself.

An external handler is an executable program. It can to perform any actions to handle the event, but it must return one of the following values:

0 : continue handlers execution
1 : stop handlers execution
2 : stop both handlers and test execution
3 : stop handlers execution, kill the test and stop tests sequence execution.

The end user, specifying full path, can use any handler programs or scripts, that are located anywhere. But if a short name is used, the handler is searched in the default handlers directory.

By default the handlers directory is RUN_DIR/handlers. The directory of handlers can be changed by user.

Configuration

Configurator

Two configuration subtrees provided to manage the serial parser framework. The first subtree is /local/tester, it is used to declare Tester events. There is also performed binding of some handlers to the events.

OID	Type	Description
/local/tester	RW none	Tester subtree.
/local/tester/enable	RW int	Enable the Tester thread to listen for events. By default: disabled.
/local/tester/period	RW int	Period for polling of events status. By default: 100 milliseconds.
/local/tester/location	RW str	Directory with handlers for short named handlers. By default: RUN_DIR/handlers.
/local/tester/event	RC none	Subtree of a Tester event.
/local/tester/event/handler	RC str	Event handler. Use a path to the executable as a value.
/local/tester/event/handler/priority	RW int	Handler priority. By default: 0.
/local/tester/event/handler/signal	RW int	Signal number. By default: SIGINT. The list of supported signals: SIGHUP, SIGINT, SIGQUIT, SIGILL, SIGTRAP, SIGABRT, SIGBUS, SIGFPE, SIGKILL, SIGUSR1, SIGSEGV, SIGUSR2, SIGPIPE, SIGALRM, SIGTERM, SIGSTKFLT. The value ‘none’ means that the signal will not be sent.
/local/tester/event/handler/internal	RW int	Indicates that it is an internal handler. By default: external.

A new subtree of the agent is /agent/parser. It is used to configure and launch a parser thread for a serial consoles on the agent.

OID	Type	Description
/agent/parser	RC str	The parser subtree. Use a name of serial console as value.
/agent/parser/port	RW int	Conserver port corresponds to the parameters of conserver launch. By default: 3109.
/agent/parser/user	RW str	Conserver user name. By default: tester.
/agent/parser/enable	RW int	Start/stop the parser thread.
/agent/parser/interval	RW int	Intreval of polling messages from the console.
/agent/parser/reset	RW int	Reset status for all events.
/agent/parser/event	RC str	Declared in the Tester subtree as a value.
/agent/parser/event/pattern	RC str	Pattern string.
/agent/parser/event/counter	RW int	Counter of the happened event.
/agent/parser/event/status	RW int	Status of the event.
/agent/parser/logging	RW int	Enable logging of the console messages in the main log. By default: enabled.
/agent/parser/logging/level	RW str	Level of the message for logging. By default: WARN.

Configuration exmaple:

<?xml version="1.0"?>
  <history>
    <add>
      <instance oid="/local:"/>
      <instance oid="/local:/tester:/event:new_event"/>
      <instance oid="/local:/tester:/event:new_event/handler:1" value="~/work/trunk/handlers/myserialhandler"/>
      <instance oid="/local:/tester:/event:new_event/handler:2" value="ext_handler_short_name"/>
      <instance oid="/local:/tester:/event:new_event/handler:3"/>

      <instance oid="/agent:Agt_A/parser:new_parser" value="console_name"/>
      <instance oid="/agent:Agt_A/parser:new_parser/event:ag_ev" value="new_event"/>
      <instance oid="/agent:Agt_A/parser:new_parser/event:ag_ev/pattern:1" value="fooo"/>
      <instance oid="/agent:Agt_A/parser:new_parser/event:ag_ev/pattern:2" value="bar"/>
    </add>
    <set>
      <instance oid="/local:/tester:/enable:" value="1"/>
      <instance oid="/local:/tester:/period:" value="200"/>
      <instance oid="/local:/tester:/location:" value="/home/andrey/work/trunk/handlers"/>
      <instance oid="/local:/tester:/event:new_event/handler:1/priority:" value="3"/>
      <instance oid="/local:/tester:/event:new_event/handler:2/priority:" value="2"/>
      <instance oid="/local:/tester:/event:new_event/handler:3/priority:" value="1"/>
      <instance oid="/local:/tester:/event:new_event/handler:3/internal:" value="1"/>
      <instance oid="/local:/tester:/event:new_event/handler:3/signal:" value="SIGINT"/>

      <instance oid="/agent:Agt_A/parser:new_parser/port:" value="3105"/>
      <instance oid="/agent:Agt_A/parser:new_parser/user:" value="tester"/>
      <instance oid="/agent:Agt_A/parser:new_parser/interval:" value="100"/>
      <instance oid="/agent:Agt_A/parser:new_parser/logging:" value="1"/>
      <instance oid="/agent:Agt_A/parser:new_parser/logging:/level:" value="WARN"/>
      <instance oid="/agent:Agt_A/parser:new_parser/enable:" value="1"/>
    </set>
  </history>

Test API

The Test API provides the possibilities to add, delete, configure and manage the parser threads and event handlers. The complete list and detailed description functions of the Test API you can find here lib/tapi/tapi_serial_parse.h.

Test API uses a tapi_parser_id structure to configure and control a parser:

typedef struct tapi_parser_id {
    const char *ta;      /**< Test agent name */
    const char *name;    /**< The parser name */
    const char *c_name;  /**< Serial console name */
    const char *user;    /**< A user name for the conserver or NULL. In case if NULL, used default value 'tester'. */
    int port;            /**< Port of the console */
    int interval;        /**< Interval to poll data from the conserver. Use -1 for default value. */
} tapi_parser_id;

It is possible to initialize the parser id structure using the TAPI. In this case the structure should be cleaned after use, because TAPI function allocates memory. Use the tapi_serial_id_cleanup() to cleanup the id structure. By default the user field value is NULL, it means in logs will use the default user - tester. Default port is 3109, interval - 100 milliseconds. See the following example.

#include "tapi_serial_parse.h"

tapi_parser_id     *pars;

/* Initialize identifier*/
pars = tapi_serial_id_init("Agt_B", "console_name", "my_parser");

/* Cleanup identifier */
tapi_serial_id_cleanup(pars);

Test API using example:

#include "tapi_serial_parse.h"

tapi_parser_id      pars;
int                 pat_i;

pars.ta         = "Agt_A";
pars.name       = "new_parser";
pars.c_name     = "console_name";
pars.user       = NULL;
pars.port       = 3109;
pars.interval   = -1;              /* value -1 is used to set default value of the polling period */

/* Add and initialize parser elements */
tapi_serial_tester_event_add("new_event");
tapi_serial_handler_ext_add("new_event", "ext_h", 5, "ext_handler_short_name");
tapi_serial_handler_int_add("new_event", "int_h", 4, SIGINT);
tapi_serial_parser_add(&pars);
tapi_serial_parser_disable(&pars);
tapi_serial_parser_enable(&pars);
tapi_serial_logging_enable(&pars, NULL);
tapi_serial_parser_event_add(&pars, "tapi_event", "new_event");
pat_i = tapi_serial_parser_pattern_add(&pars, "tapi_event", "Segmentation fault");

/* Remove the initialized elements */
tapi_serial_parser_pattern_del(&pars, "tapi_event", pat_i);
tapi_serial_parser_event_del(&pars, "tapi_event");
tapi_serial_logging_disable(&pars);
tapi_serial_parser_del(&pars);
tapi_serial_handler_del("new_event", "ext_h");
tapi_serial_handler_del("new_event", "int_h");
tapi_serial_tester_event_del("new_event");