Duvdevan Duvdevan - 1 month ago 12
Javascript Question

How to check duplicates in two columns with JavaScript and by their values?

I have an array of data like this:

var data = [
// Columns
['Target ID', 'Source ID', 'Label'],

// Key: 1
[1, 2, 'String 1/2'],
[1, 3, 'String 1/3'],
[1, 4, 'String 1/4'],
[1, 5, 'String 1/5'],

// Key: 2
[2, 1, 'String 2/1'],
[2, 3, 'String 2/3'],
[2, 4, 'String 2/4'],
[2, 5, 'String 2/5'],

// Key: 3
[3, 1, 'String 3/1'],
[3, 2, 'String 3/2'],
[3, 4, 'String 3/4'],
[3, 5, 'String 3/5']
]


As you can see, I have duplicate items in the array above, ie:

[1, 2, 'String 1/2']


And:

[2, 1, 'String 2/1']


I can't figure out the logic to remove those kind of duplicates.

All what I have achived so far is to remove duplicated items by first key value (since the array/data is created from a CSV), ie.
[1, 1, 'String 1/2']
.

Basically my question is: how to remove duplicated items from the array above?




EDIT: Better explanation of my problem.

English is not my primary, so bare with me. The string at the index
2
in the array has nothing to do with anything - it's just a descriptor.

First item in the array is the key - in the examples I provided they are -
1, 2, 3
- thus, a second item is related to the key elsewhere in the array, thus, a relation between
1, 2
and
2, 1
is a duplicate - and I want those removed.

EDIT 2: Added the column item in the array above:

So, now I hope it's more clear -- first item in each array is the
Target ID
, second is the
Source ID
and the third one is not relavant at all.

My goal and need is to remove all items which already occur in the array, if the
Target ID === Source ID
in any of the previous arrays/items.

I have tried something dumb as:

var in_target = [];
var in_source = [];

var reformed_data = [];

for(var x in data) {
var _target = data[x][0];
var _source = data[x][1];

if(in_target.indexOf( _target ) === -1) {
if(in_source.indexOf( _source) === -1) {
reformed_data.push( data );
}
}

in_target.push( _target );
in_source.push( _source );
}

console.log( reformed_data );


But is doesn't work and it's terribly slow, since I actually have a large piece of data.

Answer

I can't figure out the logic to remove those kind of duplicates.

Try this approach

var arr = [
    [1, 2, 'String 1/2'],
    [1, 3, 'String 1/3'],
    [1, 4, 'String 1/4'],
    [1, 5, 'String 1/5'],
    [2, 1, 'String 2/1'],
    [2, 3, 'String 2/3'],
    [2, 4, 'String 2/4'],
    [2, 5, 'String 2/5'],
    [3, 1, 'String 3/1'],
    [3, 2, 'String 3/2'],
    [3, 4, 'String 3/4'],
    [3, 5, 'String 3/5']
];
var map = {}; //initialize this map
var output = arr.filter( function(item){ 
  var key1 = item[0], key2 = item[1]; 
  map[key1] = map[key1] || {};  //create a map of map[1] for chain of 1
  map[key2] = map[key2] || {};  //create a map of map[2] for chain of 2
  map[key1][key2] = true;  //now mark the map[1][2] to true
  return !map[ key2 ][ key1 ] ;  //check if map[2][1] is not true
});
console.log( output );

Comments