Vinay Vinay - 1 month ago 19
Java Question

Simple Date filter in Cascading

I want to implement a simple date filter and I feel it is not as easy as I thought it is.

DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
Date date = new Date();
String datestring = dateFormat.format(date);

ExpressionFilter dateFilter = new ExpressionFilter("datefield1 <= datestring", String.class);
inputPipe = new Each(inputPipe,dateFilter);


datefield1
is a field in the
inputPipe
which I want to filter on based on the current date. The problem with the above code is that it expects to find the fields mentioned in the
ExpressionFilter
to be present in the
inputPipe
.
datestring
is not a field in the
inputPipe
and hence it is failing there.

Also tried this way but it throws a compile error. I'm new to Cascading and Java, so please excuse if I miss anything.

ExpressionFilter dateFilter = new ExpressionFilter("datefield1 <= "+datestring, String.class);

Answer

You can look at the cascading Filter option

Create a filter like following

public class DateFilter extends BaseOperation implements Filter {
    private String dateStr;
    public DateFilter(String dateStr) {
        this.dateStr = dateStr;
    }

    public boolean isRemove( FlowProcess flowProcess, FilterCall filterCall ) {
        // get the arguments TupleEntry
        TupleEntry arguments = filterCall.getArguments();

        // initialize the return result
        boolean isRemove = false;

        String inputStr = argument.getString("datefield1"); // Get the date from datefield1 field

        isRemove = compareDate(inputStr, dateStr);

        return isRemove;
    }

    private boolean compareDate(String inputStr, String dateStr) {
        // Add you logic to match the date. Try [joda](http://www.joda.org/joda-time/)
        return false;
    }
}

Once you have the filter, use it in your code like:

DateFormat dateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
Date date = new Date();
String datestring = dateFormat.format(date);

inputPipe = new Each(inputPipe, new DateFilter(datestring));

This should help you.

Reference: