Step 7 - Build the Final Output Pipeline
🔍 What You’re Doing
Section titled “🔍 What You’re Doing”This is the final shaping step. The Augment Events Pipeline produced an enriched dataset with all the joined context, but not yet in a clean, analyst-ready schema. This Output Pipeline takes that enriched output and reshapes it to match a target schema you define using a sample CSV file (AUDIT_LOG.csv).
You upload the sample to AI Copilot as a target, paste a short prompt describing the cleanup operations you want, and AI Copilot generates the column renames, reordering, and date formatting needed to match. The result is a stable, predictable audit log dataset ready for downstream consumption such as BI tools, scheduled exports, or alerting workflows.
This pattern (sample-driven output shaping) is the fastest way to lock a Pipeline to a specific schema without writing transformation logic by hand.
📥 Before You Begin
Section titled “📥 Before You Begin”Download AUDIT_LOG.csv to your local Downloads folder. You’ll upload it as the sample target output for AI Copilot in the steps below.
✅ Summary of Steps
Section titled “✅ Summary of Steps”-
From the Workflow’s Pipelines tab, click Add to create a new Output Pipeline.
-
Enter Create Final Output in the Name field and click AI Copilot.
-
Click Add Datasets and select the Augment Events Supporting Pipeline output.
-
Click into the description text area and paste the prompt below. Drag the resize handle to expand if needed.
-
Click Upload Sample Output Data, then Choose a CSV, and select AUDIT_LOG.csv from your Downloads folder.
-
Click Upload & Preview CSV.
-
Click Generate Instructions, review the AI-generated Pipeline Build Plan, and click Approve & Build.
-
Watch the Pipeline build each transformation, then click the last step to review the final output.
📋 Prompt to Copy
Section titled “📋 Prompt to Copy”Paste this into the AI Copilot description field in step 4:
1. Rename: id->AUDIT_ID, eventDate->EVENT_DATE, eventTimeZone->EVENT_TIMEZONE, eventTypeId->EVENT_ID, tenantId->TENANT_ID, objectId->OBJECT_ID, message->MESSAGE, success->SUCCESS, ipAddress->IP_ADDRESS, hostName->HOST_NAME, additionalAttributes.name->ADDITIONAL_ATTRIBUTES_NAME, additionalAttributes.objectId->ADDITIONAL_ATTRIBUTES_OBJECT_ID, Event Message->EVENT_MESSAGE, Associated Object ID->ASSOCIATED_OBJECT_ID, Notes->NOTES.
2. Parse EVENT_DATE from Unix epoch milliseconds (%Q) to datetime %Y-%m-%d %H:%M:%S, UTC.
3. Keep only these columns, in this exact left-to-right order: AUDIT_ID, EVENT_DATE, EVENT_TIMEZONE, EVENT_ID, EVENT_MESSAGE, ASSOCIATED_OBJECT_ID, NOTES, USER_ID, USER_NAME, TENANT_ID, WORKSPACE_ID, WORKSPACE_NAME, MODEL_ID, MODEL_NAME, OBJECT_ID, OBJECT_TYPE, OBJECT_NAME, MESSAGE, SUCCESS, IP_ADDRESS, HOST_NAME, ACTION_ID, ACTION_NAME, ADDITIONAL_ATTRIBUTES_NAME, ADDITIONAL_ATTRIBUTES_OBJECT_ID.
