- 0 minutes to read

Flat File CSV Plugin

Use the Nodinite "Flat File CSV" plugin on Content like a comma-, or semicolon-separated messages (e.g. CSV-files).

If you need to further refine the result, please review the Formula - CSV.

The Content usually has the following characteristics:

  • Is string based
  • Has one or more lines
  • Has a specific character to separate values (a delimiter)
    • ';', ',', ... You can use any single character
  • Each line of the file is a data record. Each record consists of one or more fields, separated by the delimiter
  • Can include commented lines (to be skipped)

Use this plugin to extract values from messages with a small payload.

The Nodinite Flat File CSV Search Field Expression Type plugin can be used to get one or more unique values from the payload (Body) of Log Events (business transactions).

A Logged message comes from a Log Event that is part of the Logging feature of Nodinite. Use the Search Fields within the self-service enabled Log Views for your business.

Quick example

Let's get you started with the following simple demonstration example valid for this search field plugin. For other and more advanced examples, please scroll down on this page to the Examples section.

Input Expression Result 
ORD;101;ACME;456;Company Name ACME ;...
ORD;102;Northwind;789;Company Name Northwind;...

Number of header rows = 0
Delimiter = ;
Fields = 1
Ignore missing fields = checked
Quote = "
Comment = #
101
102
Flat file Data Flat File CSV configuration Unique values

Features

  • Extract single or multiple unique values from delimited strings (Payload/Body, Context, result from previous Formula function)
  • Use the properties of the Flat File CSV plugin to match data of interest
    • Number of header rows -- Specify the number of rows to skip at the beginning of the content. Use it to remove the header row, and/or to skip the first n rows (User defined).
    • Delimiter (character)
    • Fields (comma (,) separated list of positive integer(s), 0 based)
    • Ignore missing fields - Ignore a missing field if the plugin can't find the field. A warning alert is otherwise set for the logged event if the data is incomplete
    • Quote - (a character, i.e. '"') The quote used to escape fields
    • Comment - (a character, i.e. '#') The character used to denote a line that is commented out

Important

This plugin loads the entire message into RAM, so make sure to apply this extraction method only on messages with a small payload.

How to use

To extract values from flat files (logged messages) with CSV, you must first configure the Search Field, in this example case; The Order Id:

  • Select 'Flat file CSV' expression type plugin
  • Enter configuration for the Flat File CSV plugin
  • Finalize the remaining steps

Values for a configured Search Field are extracted either during the standard processing (new events) or from a user-initiated re-index operation. Extracted values are persisted and stored for as long as the number of days to keep the events property on the Message Type is configured.

Test Expression

You can test an expression when configuring a Search Field in the 'Test Expression' tab

  1. Enter an appropriate payload in the 'Message Body' tab
  2. Select the 'Flat File CSV' expression type plugin
    SelectPlugin
  3. Enter valid configuration Number of header rows
  • Delimiter
  • Fields
  • Ignore missing fields
  • Quote
  • Comment
  1. Review result/output, rewrite the code in step 3 until you get the data you seek

Behind the scenes; The Nodinite Logging Service is invoked and extracts the matching values. The result presents itself in the Web Client and the evaluated processing state, and the number of unique matches.

Test Expression
Valid expression with state output, unique values and total count

If the expression is either invalid or does not match any data, then the following output is presented:
No Result
Here's an example with an invalid expression, which yields no result.

Examples

graph LR subgraph "Search Fields" sf(fal:fa-magnifying-glass-plus Order Number) end subgraph "Search Field Expressions" sfe(fal:fa-flask Flat File CSV plugin) end subgraph "MessageTypes" mt1(fal:fa-file Orders) end sf --- sfe sfe ---|Expression configuration| mt1

Example sample #1: Flat File (; Delimited)

To extract the 2nd field, for example, Order ID from the following semicolon (;) separated flat file, you can use Delimiter=; and Fields=1. This configuration yields the values 101 and 102.

Message Body 

ORD;101;ACME;456;Company Name ACME ;Dieselgate valley 1;123 45;Flameburg;
ORD;102;Northwind;789;Company Name Northwind;Radiator Spring 420;543 21;Apocalypsenburg;

Flat File CSV configuration

Delimiter=; Fields=1

Flat File Example #2
Example: Extract multiple order ids from a semicolon-separated flat file.

Next Step

How to Add or manage Search Fields
How to Add or manage Log Views

Expression Type Plugins are used in Search Fields
What are Search Fields?
What are Search Field Expressions?
What are Message Types?
What are Log Views?