Bug - multiple new msg channels

General
1

There’s some bug happening in messages. Multiple new message channels and duplicates created in the past hour.

2

Hey Dave, sorry to hear about this. Our team will take a look.

3

Hi @Dave – thank you for reaching out and surfacing this issue!

I wanted to shoot you an update. Recently, while preparing for an upcoming release, we ran a one-time “backfill” process to update existing data in your workspace. This process is routine before major releases, but in this case, a system error caused duplicate messaging channels to be created.

I understand how confusing and frustrating this may have been for you and your clients. We apologize for this experience.

Scope of the incident and steps taken to mitigate:

  1. In cases where duplicate message channels were created with no messages inside them, we’re automatically removing these channels, but the process may take another 2 hours to complete.
  2. In some rare cases, duplicate channels were created with the automated welcome message sent. These channels have also been removed, but your clients may have received an extra email notification.
  3. If any duplicate channels contained additional client communication, we decided to leave them in place to avoid erroneously deleting any legitimate communications. We recommend letting clients know to use a different channel and manually removing any duplicated channels to prevent confusion.

Next steps + prevention: We’ve updated our testing protocols and safeguards to ensure this won’t happen again. Additionally, a full retrospective is in underway to identify any additional improvement opportunities.

We apologize for this experience and appreciate your patience as we worked to resolve. If you need any support moving forward, please don’t hesitate to reach out to us here or at support@copilot.app :sparkles: