sparc_spread sparc_spread - 22 days ago 12
Python Question

pandas cut with infinite upper/lower bounds

The pandas

documentation states that: "Out of bounds values will be NA in the resulting Categorical object." This makes it difficult when the upper bound is not necessarily clear or important. For example:

cut (weight, bins=[10,50,100,200])


Will produce the bins:

[(10, 50] < (50, 100] < (100, 200]]


So
cut (250, bins=[10,50,100,200])
will produce a
NaN
, as will
cut (5, bins=[10,50,100,200])
. What I'm trying to do is produce something like
> 200
for the first example and
< 10
for the second.

I realize I could do
cut (weight, bins=[float("inf"),10,50,100,200,float("inf")])
or the equivalent, but the report style I am following doesn't allow things like
(200, inf]
. I realize too I could actually specify custom labels via the
labels
parameter on
cut()
, but that means remembering to adjust them every time I adjust
bins
, which could be often.

Have I exhausted all the possibilities, or is there something in
cut()
or elsewhere in
pandas
that would help me do this? I'm thinking about writing a wrapper function for
cut()
that would automatically generate the labels in desired format from the bins, but I wanted to check here first.

Answer

After waiting a few days, still no answers posted - I think that's probably because there really is no way around this other than writing the cut() wrapper function. I am posting my version of it here and marking the question as answered. I will change that if new answers come along.

def my_cut (x, bins,
            lower_infinite=True, upper_infinite=True,
            **kwargs):
    r"""Wrapper around pandas cut() to create infinite lower/upper bounds with proper labeling.

    Takes all the same arguments as pandas cut(), plus two more.

    Args :
        lower_infinite (bool, optional) : set whether the lower bound is infinite
            Default is True. If true, and your first bin element is something like 20, the
            first bin label will be '<= 20' (depending on other cut() parameters)
        upper_infinite (bool, optional) : set whether the upper bound is infinite
            Default is True. If true, and your last bin element is something like 20, the
            first bin label will be '> 20' (depending on other cut() parameters)
        **kwargs : any standard pandas cut() labeled parameters

    Returns :
        out : same as pandas cut() return value
        bins : same as pandas cut() return value
    """

    # Quick passthru if no infinite bounds
    if not lower_infinite and not upper_infinite:
        return pd.cut(x, bins, **kwargs)

    # Setup
    num_labels      = len(bins) - 1
    include_lowest  = kwargs.get("include_lowest", False)
    right           = kwargs.get("right", True)

    # Prepend/Append infinities where indiciated
    bins_final = bins.copy()
    if upper_infinite:
        bins_final.insert(len(bins),float("inf"))
        num_labels += 1
    if lower_infinite:
        bins_final.insert(0,float("-inf"))
        num_labels += 1

    # Decide all boundary symbols based on traditional cut() parameters
    symbol_lower  = "<=" if include_lowest and right else "<"
    left_bracket  = "(" if right else "["
    right_bracket = "]" if right else ")"
    symbol_upper  = ">" if right else ">="

    # Inner function reused in multiple clauses for labeling
    def make_label(i, lb=left_bracket, rb=right_bracket):
        return "{0}{1}, {2}{3}".format(lb, bins_final[i], bins_final[i+1], rb)

    # Create custom labels
    labels=[]
    for i in range(0,num_labels):
        new_label = None

        if i == 0:
            if lower_infinite:
                new_label = "{0} {1}".format(symbol_lower, bins_final[i+1])
            elif include_lowest:
                new_label = make_label(i, lb="[")
            else:
                new_label = make_label(i)
        elif upper_infinite and i == (num_labels - 1):
            new_label = "{0} {1}".format(symbol_upper, bins_final[i])
        else:
            new_label = make_label(i)

        labels.append(new_label)

    # Pass thru to pandas cut()
    return pd.cut(x, bins_final, labels=labels, **kwargs)
Comments