TOPIC: Ignore duplicate line(s) of data

One of our customer’s system is mysteriously duplicating lines in some orders.  Their IT department is trying to find solution. But, until they do they’ve reached out to me and ask that we ignore duplicates in the order file.  I’m trying to create a .NET to identify the duplicates and change the item to ‘xxxx’ so I can filter it on the other side.  But, I’m not having any success.  Ignoring it in the .NET script would be even better.  In the attached file there are three orders with two lines each.  I want to ignore the second line for order B456.
As always, thanks for the help.

Well...I never thought of using GroupBy.  I use it all the time to combine duplicates and sum the quantities.
I guess I was over thinking it.  Thanks, I got it.

Just for your information. A better way to filter duplicate lines/contentis to use the Filter Heater with the enabled option “Filter duplicates out”, like the screenshot below.

Note: To use the GroupBy Heater for filtering duplicate line just working when the input stream is sorted based on the grouped field!
 

Thanks Robert,

That is a better option and clears up the line count issue I was having with the GroupBy.

JD