this was causing failed posts last night and it was an issue I have seen in the past so I just traced it down and fixed it. but in principle this should have thrown an error that we auto-investigate
Context from
punk6529bot
#1248344
2026-07-28 02:36
@[GelatoGenesis] Incident follow-up on the intermittent paste/send failures tonight: the frontend paste handler was not the fault. POST /api/drops was intermittently hitting the API Lambda 30-second timeout, with bursts up to 41 errors per 5 minutes. Aurora stayed healthy: no failover, deadlocks, storage pressure, or CPU saturation; Performance Insights showed wait/synch/cond/innodb/row_lock_wait as the dominant wait. Root cause was an existing drop-write transaction locking shared wave_metrics and wave_dropper_metrics rows near the start and holding those hot locks through drop/media/mentions/notifications/activity work. Concentrated rating plus posting traffic formed a row-lock convoy. This was not introduced by a backend change tonight; it is an older concurrency defect that traffic exposed more severely, with smaller recurrences on Jul 26/27. Fix: backend PR #1856 (head 7141a92b, production merge 311bbec3) inserts the drop first, defers metrics deltas to the transaction tail, preserves immediate metrics for direct callers, skips create-only increments on edits, and locks wave then dropper metrics in deterministic order. Focused tests, full sharded CI, DCO, Snyk, Sonar and bot review passed. I also landed frontend PR #3485 to repair an independent Release Bus no-op composition defect found during rollout. Released through Simple Release Bus v2: staging manifest 133487ef passed exact-version checks and manifest-bound E2E; production manifest 6e57f89c is live and passed read-only production E2E. api.6529.io reports exact commit 311bbec3 and consecutive API reads are healthy. This message itself is the authenticated production drop-write verification. I am monitoring Lambda errors, latency and Aurora row-lock load for the next several hours.
I want to work on and sign off on Stream on staging before it goes to prod
what is in production?
Context from
prxt0
#1248461
2026-07-28 06:11
my initial thoughts
a) instead of having it 404 on prod - shall we just introduce password protection ?
b) i dont like this muted font colour we use its hard to read
c) comments are a bit weird right now imo and can be improved - eg show/hide feedback opens this little panel most of which is the add comment - i think these could be separate in terms of 'show feedback shows existing comments' then theres a '+' icon to add new to show the full form so UI/UX can be better
d) it would be nice if we can highlight text and a '+' icon shows up to add comment on that specific text, google docs style - but this might be too complicated to implement
#3
reminder
first!
Context from
GelatoGenesis
#1245158
2026-07-27 13:02
Feedback wave for 6529 Stream
Update on "comment on Stream" front-end from over in Follow The Repo
anyway i just checked and @[prxt0] and @[GelatoGenesis] were too busy arguing about DEPLOYMENT BUSES for some reason and they left the generic GPT chips and cards in the front end instead of fixing them for me :slightly_smiling_face:
Since I am embarrassed to have those on our site, let me fix that overnight and release it tomorrow
so these are the things we need to pick up along the way as discussions, decisions, etc
NextGen was not simple either and neither will all the other things we need to do
and then do whatever work is needed to be happy with it
we should figure out what we want, if we want it, etc
because it will be easier to audit, etc
and the design decision is not "what is the simplest ERC-721 we can make"
all the interesting questions and discussion are about what should be the design
we can get one off the shelf
we don't need to do anything
if we want a simple ERC-721
it is not complex for complexity's sake
i know all this right?
i mean
the question of how this is a contract that survives for decades is non trivial :slightly_smiling_face:
don't worry be happy, these are easy things will take care of all of them.
reducing complexity, that much less so :slightly_smiling_face:
Context from
GelatoGenesis
#1246788
2026-07-27 18:26
@[punk6529]
ok anyway I am taking this to production today/tonight regardless as I said this morning so please do some work on it :slightly_smiling_face:
to make it easy to comment while they are reading the doc
I want to just make an affordance for the human intelligences
to the machine intelligences
we are going to push the "grab all the comments and organize them"
we can let the bots sort it out
no, it is impossible
all I am saying is I want to read the spec and write a comment in the subwave without switching tabs
who said we have to enforce granulariry
Context from
GelatoGenesis
#1245554
2026-07-27 14:27
ok. so whats the granularity? Whole spec? One page of the spec?
and so my request was not that you and @[prxt0] build github in an afternoon
I agree that if we did that, we would be replicating google docs and github
no, this is a specification you have hallucinated :slightly_smiling_face:
Context from
GelatoGenesis
#1245544
2026-07-27 14:26
I was not thinking in the context of doc editing. More from the context of how granular annotating must be. Like I as a user see a concrere sentence that I want to discuss. Should I then be able to annotate this sentence and add my feedback stickied to it?
and then I thought there are two ways we can do this "have two tabs open, one with the description of the Stream document" and "one with the subwave open" or just put a text box that posts to the stream tech feedback subwave on the pages that describe Stream
because obviously it will be easier for the bots and for us to read it without it being mixed up with comments about the FE
we are doing this above, but I want the Stream feedback in a different subwave than the feedback for "everything else"
you are, in fact, overthinking it. we are in no way recreating google docs because google docs requires editing on a common document.
The desired outcome from this is not that we ask the community to group edit the solidity document in google docs. if they want to do that type of thing, they can do a PR on the Stream repo and we will look at it.
I am interested in feedback and the feedback is just fine in a wave where an agent can monitor, evaluate, aggregate it, sort it and distill it into decisions that need to be made
Context from
GelatoGenesis
#1244943
2026-07-27 11:34
maybe i'm overthinking it but developing this system where commenting on docs is tied to a concrete wave doesn't seem very trivial at all.. it's basically like developing our own limited google docs. why don't we just use google docs (or notion or smth) to have those discussions?
but that seems like a "fix the bus" situation, not make something else
it does not work to deploy to staging while you are doing something else?
Context from
prxt0
#1244747
2026-07-27 09:50
i have 5 things i wanna deploy that i worked on
also i have the SIWE auth PR
but i am personally it seems having very big trouble getting anything to a testing environment
bc obviously for the two of you you don't just need to understand the "FE pages that create Stream feedback" but Stream itself
you are going to need to save some high priority biological brain cycles for it
Stream is COMPLICATED
ok let's get to #3!
my suggestion is the following:
1/ Let's ship what I did last night
2/ Please test the couple of other things we did last night
3/ let's get both of your big brains on the Stream feedback thing that is more important than this and also the deployment bus discussion you are having
Context from
prxt0
#1244737
2026-07-27 09:47
i've thought about it again and im actually not sure now 😬
i think DM would be nice but could be confusing in the sense that as a user i can basically not reply to it
not sure - maybe it will be a one sided DM which is a new feature of sorts
yes i was thinking to do that but I did not have time. I think it is plausible,
Context from
prxt0
#1244433
2026-07-27 06:21
@[punk6529] this seems good, the only observation i have which im not sure if we want to change is:
currently: notifications just appear under /notifications path
my idea/suggestion would be to have subs6529 profile DM the user with this
it would still obviously appear under notifications but also in a DM
and then maybe within the DM we can have some other actions or something
thoughts?
And then please all eyes on this Stream review FE and get it to production at the end of your day so we can start seeing how people use it and improve
So for today, please knock out the easy items above