Custom Connector for Mule ESB Logging

Understanding Mule ESB Logging with Nodinite

The Nodinite Custom Connector enables centralized, structured logging from Mule ESB flows by replacing standard Logger and Business Events components. This guide explains the architecture, implementation steps, and configuration requirements for production-grade logging from Mule ESB 3.6–3.9 environments.

Why Use a Custom Connector for Logging

Mule ESB provides built-in logging options (Logger, Business Events), but these have significant limitations for enterprise integration scenarios:

Comparison: Built-in Logging vs. Custom Connector

Architecture: How the Custom Connector Works

Architecture: Mule ESB flows invoke the Custom Connector subflow, which formats JSON Log Events and posts them to a configured destination (queue, database, or folder). The Pickup Service asynchronously fetches events and forwards them to the Log API.

Key Components

Why Asynchronous Logging?

Asynchronous logging (via queue or database) is strongly recommended for production environments because:

• Non-blocking: Flow execution continues without waiting for log persistence
• Resilience: Destination outages don't block integration flows
• Scalability: Queue-based pickup handles high-volume bursts
• Performance: No synchronous HTTP calls from Mule flows

Destination Options

Implementation Steps

Prerequisites

Before implementing the Custom Connector, ensure you have:

• Mule ESB 3.6, 3.7, 3.8, or 3.9 installed
• Anypoint Studio 6 or 7
• Nodinite portal account for connector download
• Destination middleware configured (ActiveMQ, PostgreSQL, or file folder)
• Pickup Log Events Service Logging Agent installed and configured
• Understanding of Asynchronous Logging principles

Step 1: Download the Custom Connector

1. Log in to the Nodinite portal
2. Navigate to Downloads → Logging Agents → Mule ESB Custom Connector
3. Download the ZIP file containing the JAR package

Step 2: Add Custom Connector to Anypoint Studio

For Anypoint Studio 6

1. Extract the ZIP file
2. Copy the JAR file to your Anypoint Studio project's lib folder
3. The connector appears in the palette as Nodinite (prefix Log ...)

Screenshot: Nodinite Custom Connector installed in Anypoint Studio 6 palette.

For Anypoint Studio 7

1. Right-click project → Import → General → File System
2. Browse to extracted JAR location
3. Select JAR and click Finish

Screenshot: Import dialog for Anypoint Studio 7 JAR package.

Step 3: Create the Logging Subflow

Create a reusable logging subflow that all integration flows will invoke:

1. Create a new subflow named nodinite-logging-subflow
2. Add the following components:
   • Nodinite: Create Log Event - formats the JSON Log Event
   • Destination connector - posts event to chosen destination (HTTP for Log API, JMS for ActiveMQ, etc.)
   • Set Payload - restores original payload after logging

Screenshot: Reusable logging subflow with Create Log Event and destination components.

Example subflow configuration (ActiveMQ destination)

<sub-flow name="nodinite-logging-subflow">
    <logger message="#[sessionVars.nodiniteLogText]" level="INFO" doc:name="Logger"/>
    <set-variable variableName="payloadBeforeLogging" value="#[payload]" doc:name="Store Payload"/>
    
    <nodinite:create-log-event config-ref="Nodinite__Configuration" 
        endPointName="#[flowVars.endpointName]" 
        endpointUri="#[flowVars.endpointUri]" 
        endPointDirection="#[sessionVars.nodiniteEndPointDirection]" 
        originalMessageTypeName="#[sessionVars.nodiniteMessageTypeName]" 
        logStatus="#[sessionVars.nodiniteLogStatus]" 
        logText="#[sessionVars.nodiniteLogText]" 
        payload="#[payload]" 
        doc:name="Create JSON Log Event">
        <nodinite:context-properties ref="#[sessionVars.nodiniteContextValues]"/>
    </nodinite:create-log-event>
    
    <jms:outbound-endpoint queue="nodinite.log.events" connector-ref="ActiveMQ" doc:name="Post to ActiveMQ"/>
    
    <set-payload value="#[flowVars.payloadBeforeLogging]" doc:name="Restore Payload"/>
</sub-flow>

Step 4: Invoke the Logging Subflow from Integration Flows

Add logging calls at key points in your integration flows:

1. Before transformation - log incoming message
2. After transformation - log outgoing message
3. Exception handlers - log error conditions
4. Conditional branches - log routing decisions
5. Subflow entry/exit - log component boundaries

Example: File-based integration flow with logging

Screenshot: Mule ESB flow using Nodinite Custom Connector at receive, before processing, and after processing steps.

<flow name="invoice-processing-flow">
    <file:inbound-endpoint path="/drop/invoices" />
    
    <!-- Log: File Received -->
    <scripting:component doc:name="Set Logging Variables">
        <scripting:script engine="Groovy"><![CDATA[
            sessionVars['correlationId'] = message.id;
            sessionVars['nodiniteMessageTypeName'] = "Invoice#1.0";
            sessionVars['nodiniteLogText'] = "File Received";
            sessionVars['nodiniteLogStatus'] = 0;
            sessionVars['nodiniteEndPointDirection'] = 0;
            sessionVars['nodiniteContextValues'] = [
                fileName: flowVars['originalFilename'],
                fileSize: payload.length
            ];
            return payload;
        ]]></scripting:script>
    </scripting:component>
    <flow-ref name="nodinite-logging-subflow" doc:name="Log Receive"/>
    
    <!-- Process message -->
    <transformer ref="invoiceTransformer"/>
    
    <!-- Log: Processing Complete -->
    <scripting:component doc:name="Update Logging Variables">
        <scripting:script engine="Groovy"><![CDATA[
            sessionVars['nodiniteLogText'] = "Processing Complete";
            sessionVars['nodiniteLogStatus'] = 0;
            sessionVars['nodiniteEndPointDirection'] = 1;
            sessionVars['nodiniteContextValues'] = [
                fileName: flowVars['originalFilename'],
                invoiceNumber: flowVars['invoiceNumber'],
                amount: flowVars['invoiceAmount']
            ];
            return payload;
        ]]></scripting:script>
    </scripting:component>
    <flow-ref name="nodinite-logging-subflow" doc:name="Log Success"/>
    
    <file:outbound-endpoint path="/processed/invoices"/>
</flow>

Step 5: Configure the Pickup Service

The Pickup Log Events Service Logging Agent fetches events from your configured destination and forwards them to the Log API.

Configuration steps

1. Install the Pickup Service on a server with access to your destination middleware
2. Configure the pickup source (ActiveMQ queue, PostgreSQL table, or file folder)
3. Configure the Log API target endpoint
4. Set the pickup interval (recommend 30-60 seconds for production)
5. Configure retry and error handling policies

See the Asynchronous Logging and Pickup Service documentation for detailed configuration examples.

Step 6: Verify Logging and Fine-Tune

1. Deploy and test:

   • Deploy your updated Mule flows
   • Trigger a test message
   • Verify event appears in Nodinite Web Client

2. Review log quality:

   • Check that Message Types are correctly set
   • Verify Search Fields extract expected values
   • Confirm Log Status Codes reflect business outcomes

3. Fine-tune performance:

   • Adjust destination batch sizes
   • Optimize Pickup Service intervals
   • Monitor destination queue/table depths

JSON Log Event Format

The Custom Connector formats Mule messages as JSON Log Events following the Nodinite schema. Understanding this format is critical for proper integration.

Mandatory Fields

The following fields are required for every logged event:

Optional But Recommended Fields

See the JSON Log Event documentation for the complete schema and examples.

Message Types Configuration

Important

Message Types are mandatory and cannot be changed after logging. The OriginalMessageTypeName field determines which Search Field Expressions are applied to extract business data from logged events.

Why Message Types Matter

Many Nodinite features depend on well-defined Message Types:

1. Search Fields - Extract Order Numbers, Customer IDs, Invoice amounts, Transaction codes using Search Field Expressions bound to Message Types
2. Log Views - Filter and search logs by business identifiers (not just timestamps and correlation IDs)
3. Non-Events Monitoring - Track message volumes, detect missing messages, alert on pattern anomalies
4. Business Process Modeling (BPM) - Correlate transactions end-to-end across Mule flows and other integration platforms

Without proper Message Types

• Search Field extraction fails - no business data in search results
• Log Views can only filter by technical metadata
• Non-Events cannot detect missing invoices, orders, or payments
• BPM cannot correlate multi-step business processes

Naming Best Practices

Use consistent, meaningful Message Type names:

• Include domain and version: Invoice#1.0, Order#2.0
• Use PascalCase: CustomerUpdate#1.0 (not customer_update_v1)
• Version changes: Increment version when payload schema changes significantly
• Avoid generic names: Use SAPPurchaseOrder#1.0 instead of Message#1.0

Example: Setting Message Type in Mule flow

// Set Message Type in session variable before logging
sessionVars['nodiniteMessageTypeName'] = "Invoice#1.0";

Planning Message Types During Development

Message Types cannot be changed after events are logged. Plan your naming convention during development:

1. Identify message types in your integration flows (orders, invoices, confirmations, etc.)
2. Define naming convention for your organization
3. Create Search Field Expressions for each Message Type before go-live
4. Document Message Types in your integration design specifications

Context Options and Custom Properties

Context Options allow you to add custom key-value pairs to logged events for filtering, searching, and reporting.

Common Context Properties

Example: Setting context properties in Groovy

sessionVars['nodiniteContextValues'] = [
    orderId: flowVars['orderId'],
    customerId: flowVars['customerId'],
    invoiceAmount: flowVars['amount'].toString(),
    currency: "USD",
    isGDPRData: "true"
];

Repair and Resubmit Properties

Use context properties to enable message repair and resubmission:

sessionVars['nodiniteContextValues'] = [
    orderId: flowVars['orderId'],
    // Repair/resubmit metadata
    originalEndpoint: "file:///drop/invoices",
    retryCount: flowVars['retryCount'].toString(),
    canResubmit: "true",
    resubmitEndpoint: "vm://reprocess-invoice"
];

Build your Mule flows to handle resubmitted messages (check retryCount, implement idempotency, etc.).

Exception Handling and Error Logging

Add the Custom Connector to exception handlers to log error conditions with appropriate Log Status Codes:

<flow name="invoice-processing-flow">
    <file:inbound-endpoint path="/drop/invoices" />
    
    <!-- Normal processing ... -->
    
    <catch-exception-strategy doc:name="Catch Exception Strategy">
        <scripting:component doc:name="Set Error Logging Variables">
            <scripting:script engine="Groovy"><![CDATA[
                sessionVars['nodiniteMessageTypeName'] = "Invoice#1.0";
                sessionVars['nodiniteLogText'] = "Processing failed: " + exception.getMessage();
                sessionVars['nodiniteLogStatus'] = -1; // Error status
                sessionVars['nodiniteEndPointDirection'] = 0;
                sessionVars['nodiniteContextValues'] = [
                    fileName: flowVars['originalFilename'],
                    errorType: exception.getClass().getSimpleName(),
                    errorMessage: exception.getMessage(),
                    stackTrace: exception.getStackTrace().toString()
                ];
                return payload;
            ]]></scripting:script>
        </scripting:component>
        <flow-ref name="nodinite-logging-subflow" doc:name="Log Error"/>
    </catch-exception-strategy>
</flow>

Log Status Code Guidelines

• -1 (Error) - Processing failure, message rejected, exception thrown
• 0 (Success) - Normal processing, message accepted
• 1 (Information) - Audit trail, milestone reached, no action required

Correlation

Correlation links related log events across flows, systems, and time. See the Mule ESB Correlation guide for detailed correlation strategies.

Basic correlation example

// Set correlation ID at flow entry
sessionVars['correlationId'] = message.id; // or flowVars['orderId']
message.correlationId = sessionVars['correlationId'];

// Use same correlation ID for all log events in this flow
sessionVars['nodiniteContextValues'] = [
    correlationId: sessionVars['correlationId']
];

Log Views and Self-Service

Create role-based Log Views to empower business users:

1. Finance Team - search by invoiceNumber, customerId, invoiceAmount
2. Operations Team - search by orderId, shipmentId, warehouse
3. Support Team - search by errorType, customerComplaint, ticketNumber

This reduces incidents by enabling self-service troubleshooting.

Best Practices Summary

• Start small - implement in 2-3 flows before rolling out organization-wide
• Use ActiveMQ for production (best scalability and fail-over)
• Plan Message Types during development - they cannot be changed after logging
• Create Search Field Expressions for each Message Type before go-live
• Log at key milestones - receive, transform, send, error
• Use meaningful Log Text - help users understand what happened
• Add context properties - enable business-level searching and filtering
• Implement correlation - link related events across flows and systems
• Test pickup service - verify events reach Nodinite Web Client
• Create Log Views - empower business users with self-service access

Next Steps

Add or manage Search Fields
Add or manage Log Views
Mule ESB Correlation
Pickup Log Events Service
Asynchronous Logging

Related Topics

Logging
Log Events
Message Types
Search Fields
Search Field Expressions
Log Views
Non-Events Monitoring
Business Process Modeling (BPM)
Log Status Codes
Context Options
Endpoints
Endpoint Directions
Endpoint Types[]
JSON Log Event
Log API
Log Agents

Example: Complete Mule Flow Configuration

<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:http="http://www.mulesoft.org/schema/mule/http"
      xmlns:vm="http://www.mulesoft.org/schema/mule/vm"
      xmlns:file="http://www.mulesoft.org/schema/mule/file"
      xmlns:jms="http://www.mulesoft.org/schema/mule/jms"
      xmlns:scripting="http://www.mulesoft.org/schema/mule/scripting"
      xmlns:nodinite="http://www.mulesoft.org/schema/mule/nodinite"
      xmlns="http://www.mulesoft.org/schema/mule/core" 
      xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
      xmlns:spring="http://www.springframework.org/schema/beans"
      version="CE-3.8.1">

    <!-- Configuration -->

    <nodinite:config name="Nodinite__Configuration" logAgentValueId="33" doc:name="Nodinite: Configuration"/>

    <spring:beans>
        <spring:bean id="ActiveMQ" class="org.apache.activemq.ActiveMQConnectionFactory">
            <spring:property name="brokerURL" value="tcp://10.0.0.10:61616"/>
        </spring:bean>
    </spring:beans>

    <jms:activemq-connector name="ActiveMQ" specification="1.1" brokerURL="tcp://10.0.0.10:61616" validateConnections="true" doc:name="Active MQ"/>

    <!-- Logging Sub Flow -->

    <sub-flow name="nodinite-logging-subflow">
        <logger message="#[sessionVars.nodiniteLogText]" level="INFO" doc:name="Logger"/>
        <set-variable variableName="payloadBeforeLogging" value="#[payload]" doc:name="Store Payload"/>
        
        <nodinite:create-log-event config-ref="Nodinite__Configuration" 
            endPointName="#[flowVars.endpointName]" 
            endpointUri="#[flowVars.endpointUri]" 
            endPointDirection="#[sessionVars.nodiniteEndPointDirection]" 
            endPointTypeId="60"
            originalMessageTypeName="#[sessionVars.nodiniteMessageTypeName]" 
            logStatus="#[sessionVars.nodiniteLogStatus]" 
            logText="#[sessionVars.nodiniteLogText]" 
            payload="#[payload]" 
            doc:name="Create JSON Log Event">
            <nodinite:context-properties ref="#[sessionVars.nodiniteContextValues]"/>
        </nodinite:create-log-event>
        
        <jms:outbound-endpoint queue="nodinite.log.events" connector-ref="ActiveMQ" doc:name="Post to ActiveMQ"/>
        
        <set-payload value="#[flowVars.payloadBeforeLogging]" doc:name="Restore Payload"/>
    </sub-flow>

    <!-- Main Flow -->

    <flow name="nodinite-mule-flow" doc:name="INT001: Invoice Processing">
        <file:inbound-endpoint path="/drop/invoices" responseTimeout="10000" doc:name="File" moveToDirectory="/processed/invoices"/>
        <set-variable variableName="endpointName" value="INT001: Receive Invoices" doc:name="Set Endpoint Name"/>
        <set-variable variableName="endpointUri" value="file:///drop/invoices" doc:name="Set Endpoint URI"/>
        
        <!-- Log: File Received -->
        <scripting:component doc:name="Set Logging Variables">
            <scripting:script engine="Groovy"><![CDATA[
                sessionVars['correlationId'] = message.id;
                sessionVars['nodiniteMessageTypeName'] = "Invoice#1.0";
                sessionVars['nodiniteLogText'] = "File Received";
                sessionVars['nodiniteLogStatus'] = 0;
                sessionVars['nodiniteEndPointDirection'] = 0;
                sessionVars['nodiniteContextValues'] = [
                    fileName: flowVars['originalFilename'],
                    fileSize: payload.length.toString(),
                    correlationId: sessionVars['correlationId']
                ];
                return payload;
            ]]></scripting:script>
        </scripting:component>
        <flow-ref name="nodinite-logging-subflow" doc:name="Log Receive"/>
        
        <!-- Transform message (your business logic here) -->
        <set-payload value="#['Invoice processed: ' + payload]" doc:name="Transform"/>
        
        <!-- Log: Processing Complete -->
        <scripting:component doc:name="Update Logging Variables">
            <scripting:script engine="Groovy"><![CDATA[
                sessionVars['nodiniteLogText'] = "Processing Complete";
                sessionVars['nodiniteLogStatus'] = 0;
                sessionVars['nodiniteEndPointDirection'] = 1;
                sessionVars['nodiniteContextValues'] = [
                    fileName: flowVars['originalFilename'],
                    invoiceNumber: "INV-12345",
                    amount: "1250.00",
                    currency: "USD",
                    correlationId: sessionVars['correlationId']
                ];
                return payload;
            ]]></scripting:script>
        </scripting:component>
        <flow-ref name="nodinite-logging-subflow" doc:name="Log Success"/>
        
        <file:outbound-endpoint path="/processed/invoices" responseTimeout="10000" doc:name="File"/>
        
        <!-- Exception Handler -->
        <catch-exception-strategy doc:name="Catch Exception Strategy">
            <scripting:component doc:name="Set Error Logging Variables">
                <scripting:script engine="Groovy"><![CDATA[
                    sessionVars['nodiniteMessageTypeName'] = "Invoice#1.0";
                    sessionVars['nodiniteLogText'] = "Processing failed: " + exception.getMessage();
                    sessionVars['nodiniteLogStatus'] = -1;
                    sessionVars['nodiniteEndPointDirection'] = 0;
                    sessionVars['nodiniteContextValues'] = [
                        fileName: flowVars['originalFilename'],
                        errorType: exception.getClass().getSimpleName(),
                        errorMessage: exception.getMessage(),
                        correlationId: sessionVars['correlationId']
                    ];
                    return payload;
                ]]></scripting:script>
            </scripting:component>
            <flow-ref name="nodinite-logging-subflow" doc:name="Log Error"/>
        </catch-exception-strategy>
    </flow>
</mule>