A client-side duplicate message filter

[Makaze]

I and other users with intermittent connections often receive strings of duplicate messages in the IP.Chat application despite any filtering on the server side. It gets so bad that new messages become impossible to track as they are drowned out by floods of old messages.

On behalf of users with slow connections, please add a client-side filter to your application. Doing so would eliminate duplication for users regardless of their connection and create a more perfect interface for all who enjoy your application. As an amateur I believe that adding unique IDs to each message element and then filtering for those IDs before generating new messages will solve this problem.

Thank you for your consideration.

[bfarber]

Hmm, this already happens and I can't say I've ever actually seen any duplicate messages show up in the chat application for me.

Behind the scenes, IP.Chat reads the ID and only shows messages with newer IDs when it downloads from the server.

[Makaze]

Hmm, this already happens and I can't say I've ever actually seen any duplicate messages show up in the chat application for me.

Behind the scenes, IP.Chat reads the ID and only shows messages with newer IDs when it downloads from the server.

There are no IDs on the messages on my end. See:

<div id="messages-display" class="messages-list">
<ul id="storage_chatroom">
<li class="chat-time">
<div>
2:05 PM
</div>
</li>
<li class="post chat-notice">
<img class="ipsUserPhoto ipsUserPhoto_mini" src="http://community.invisionpower.com/filestore/uploads/profile/photo-519079.png?_r=1386984074">
 
<label>
Makaze
</label>
 
<div>
  has entered the room
</div>
</li>
<li class="post chat-message chat-myown">
<img class="ipsUserPhoto ipsUserPhoto_mini" src="http://community.invisionpower.com/filestore/uploads/profile/photo-519079.png?_r=1386984074">
 
<label>
Makaze
</label>
 
<div>
This is a test message.
</div>
</li>
<li class="post chat-message ">
<img class="ipsUserPhoto ipsUserPhoto_mini" src="http://serenesforest.net/forums/uploads/profile/photo-thumb-9385.jpg?_r=1386771038">
 
<label>
<span style="color:green">
<u>
PandaCelt
</u>
</span>
</label>
 
<div>
Once, a guy said I reminded him of one of the Hangover characters.
</div>
</li>
<li class="post chat-notice">
<img class="ipsUserPhoto ipsUserPhoto_mini" src="http://community.invisionpower.com/filestore/uploads/profile/photo-thumb-157477.png?_r=1372435927">
 
<label>
<span style="color:#FF0000;font-weight:bold;" title="S">
+
</span>
Andrej
</label>
 
<div>
  has entered the room
</div>
</li>
</ul>
</div>

[bfarber]

No, I said behind the scenes. I'm not talking about an HTML ID, I'm talking about an ID the server sends which the javascript code "sees".

The chat application remembers the last time it got an update in the javascript file, and sends that with the requests to only retrieve messages newer than this timestamp.

[Makaze]

It doesn't do so if the connection is bad. It generates the messages but does not recognize that it has. I tested it last night and got duplicates in both Firefox and Chrome on separate chats. One of them was the demo here.

Chrome:



Firefox:



HTML IDs will be much more to the point regarding this problem.

[bfarber]

That's pretty strange - this would be the first report of this issue I've seen that I can recall.

Given that you saw that here, it would seem to be connectivity issues on your end and not something we could duplicate if you submitted a ticket, which makes it even more challenging to try to debug.

IP.Chat is getting a complete overhaul (more to come on that) for 4.0, and the way it is being rearchitected will likely remove this (potential) issue.

HTML IDs will be much more to the point regarding this problem.

HTML IDs aren't the problem - I wouldn't focus on this at all (it's not where the issue is coming in to play).

[Makaze]

Correct me if I am wrong: The problem is that the server cannot tell that the messages have been generated already. This is because it does not check the messages that have been generated; instead, it checks the last load time or what have you (this what what you seemed to be saying). For some reason, it generates the messages and then fails to log it so that the server can check when it downloads the next set.

I say that HTML IDs will be more to the point because if you use HTML IDs, you do not need the "behind the scenes" (whatever that means) filter. It will become redundant. An inline ID system will solve the issue without the need for a secondary background system. An HTML ID solution will also work independent of any connection problems.

I am aware that the current system does not use HTML IDs. My question is why doesn't it? Why wouldn't it be preferable?

[bfarber]

It does not add any value to use HTML ids. There is no reason for or against it, I'm merely stating it doesn't inherently solve the problem. You CAN insert the same HTML ID more than once into a document (even if you shouldn't), so you still need code to check that the HTML ID doesn't exist in the document...so by that measure, we could just change the method we already have in place to account for the situation you are running in to. One possibility might be to store this timestamp before making the call rather than afterwards, as based on what you describe it sounds like the first call to retrieve messages has not yet finished by the time the second one starts, which is why the same messages are pulled more than once.

Granted, without being able to duplicate and the fact no one else is reporting this makes this a bit tricky to investigate, but all I was saying was that using (or not using) HTML IDs themselves won't solve this issue.

[Makaze]

The solution you proposed sounds problematic because if you log the time when it starts, you end up assuming that it will succeed and filter out messages that never make it back.

Main problem: If it checks the timestamps of calls instead of the IDs of the messages already generated, it doesn't matter when you log the timestamp because the options are between sitting and doing nothing until a call goes through, or sending a second call. It will always be possible for the client's internet to be intermittent enough that multiple calls are sent to the server and thus multiple calls finish. You need to filter messages based on the messages, not the time, such that when the client receives identical calls at different times, it checks the individual messages already generated instead of or on top of the calls made and received.

To put it another way, time based filters are essentially server-based because they rely on the speeds of transfers between the client and the server. Using server response time to filter on the client-side is counterproductive to making it client-side. That was the purpose of HTML IDs and checking them: a purely client-side filter.

If you need me to test something on my end, I am willing, but I imagine you would prefer to do it yourselves.