Shankar Shankar - 1 year ago 154
Scala Question

Spark SQL - IN clause

I would like to add where condition for a column with Multiple values in DataFrame.

Its working for single value, for example.

df.where($"type".==="type1" && $"status"==="completed").

How can i add multiple values for the same column like below.

df.where($"type" IN ("type1","type2") && $"status" IN ("completed","inprogress")

Answer Source

the method you are looking for is isin:

df.where($"type".isin("type1","type2") and $"status".isin("completed","inprogress"))
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download