tyrell_c tyrell_c - 1 month ago 8
Node.js Question

cb is not a function when trying to use a different parser

I have s3 server access logs in a bucket and I'm trying to get them to ElasticSearch service on AWS with a lambda function.

The example lambda function that I'm using, which uses

clf-parser
to paste apache common log files. Since I want to use s3-log-parser, I have made the following changes:

// instead of
var parse = require('clf-parser');
// I have
var s3logparser = require('s3-log-parser');

// instead of
var logRecord = parse(line.toString());
// I have
var logRecord = s3LogParser.parse(line.toString());`


and I get

ReferenceError: s3LogParser is not defined


am I calling the module wrong? I did
var s3LogParser= require('s3-log-parser');
to fix that error and now I get
TypeError: cb is not a function


I noticed that in index.js for s3 parser there is this line :
cb(null, parsedLogs);
... trying to figure out how to fix this callback issue ...

Answer

It appears as though the s3-log-parser module expects a callback and doesn't return anything from its parse() function even though the function is 100% synchronous. So the only way to get your parsed logs is to provide a callback function.

var logRecord = s3LogParser.parse(line.toString(), function (err, lines) {
  logRecord = lines
})
console.log(logRecord)

Edit

Using s3-access-log-parser instead (example):

var s3alp = require("s3-access-log-parser")
var bogusCharacters = new RegExp(String.fromCharCode(8204, 8203), 'g')
var logRecord = s3alp(line.toString().replace(bogusCharacters, ''))