Merging upstream into Catodon

Catodon is a fork of Sharkey, which is a fork of Misskey. Changes therefore travel down a chain: Misskey to Sharkey, Sharkey to Catodon.

This document is the checklist for that merge. It is inherited from Sharkey's own procedure for merging Misskey. Paths for Catodon's counterparts have been checked in this repository; some upstream source paths deliberately do not exist here because Catodon deleted or replaced those files. What the checklist has not had is a full rehearsal under Catodon: treat it as a well-informed starting point rather than a proven runbook, and expect to discover Catodon-specific conflicts that nobody has written down yet. Add them here when you do.

The merge itself

Add the upstream remote to your clone once:

git remote add sharkey https://activitypub.software/TransFem-org/Sharkey.git

Then:

git remote update
git checkout develop
git checkout -b merge/$(date +%Y-%m-%d)
git merge --no-ff sharkey/develop

Resolve the conflicts and commit that merge before doing anything else. Conflicts in pnpm-lock.yaml usually resolve by running pnpm install, which detects the markers and does a decent job.

Everything below happens on the same branch, in commits of its own, after the merge commit exists.

Carrying changes into Catodon's parallel code

Catodon and Sharkey both keep code that upstream has only one copy of. A change that upstream made in one place has to be copied by hand into each of its counterparts here. When Catodon deleted the upstream source file, inspect it with git show sharkey/develop:<path>. Commit after each step.

Upstream map is the measured version of this list, against Misskey: where each upstream path lives here, how far it drifted, which files here copy it, and the Catodon-only paths an upstream diff never names.

In packages/backend/src/core/activitypub/models/ApNoteService.ts, from createNote into updateNote.
From packages/backend/src/core/NoteCreateService.ts into packages/backend/src/core/NoteEditService.ts, for the blocks the edit path still writes its own way: the visibility a repost or a reply imposes, the repost blocking check, the mention of the parent's author, and the ActivityPub delivery. The stages both paths share (channel and content rules, text trimming, entity extraction, addressees, forum visibility, mentions, conversation context, timeline fan-out, counters, search indexing, the parent-author check and the quote request) are NoteCreateService methods that NoteEditService calls, and a change to them needs no copy.
From the input checks of packages/backend/src/server/api/endpoints/notes/create.ts into packages/backend/src/server/api/NoteInputService.ts, which notes/create, notes/edit and notes/schedule/create share. A rule changed there reaches all three endpoints; run test/unit/NoteInputValidation.ts and read the rows whose answer changed.
From Sharkey's Mk note components into Catodon's Sk counterparts, where it makes sense: MkNote.vue, MkNoteDetailed.vue, MkNoteSimple.vue, MkNoteSub.vue into SkNote.vue, SkNoteDetailed.vue, SkNoteSimple.vue, SkNoteSub.vue. Those Mk sources no longer exist on the Catodon side; inspect them on sharkey/develop. The post header and the server ticker are the exception: each is one file under the upstream name, MkNoteHeader.vue and MkInstanceTicker.vue, where an upstream change applies directly and a change Sharkey made to its SkNoteHeader.vue or SkInstanceTicker.vue goes too. What Catodon adds to them sits in a //#region Catodon after the upstream code, in the script and in the style.
From Misskey's packages/frontend/src/composables/use-note.ts into Catodon's packages/frontend/src/use/use-note.ts, which keeps its exports (useNote, checkNoteWordMute, checkBuiltinSoftMute and the UseNote* types) so that a hunk finds its function by name. Then check the four components that call it: SkNote.vue, SkNoteDetailed.vue, SkNoteSub.vue and SkForumPostCard.vue. So far it holds the like button, the reaction picker and the repost state; the rest of the post behaviour still lives in each component, and the bullet above still applies to it.
From the global timeline into the bubble timeline: packages/backend/src/server/api/endpoints/notes/global-timeline.ts into bubble-timeline.ts beside it, packages/backend/src/server/api/stream/channels/global-timeline.ts into bubble-timeline.ts, and the frontend side in packages/frontend/src/timelines.ts, components/MkTimeline.vue, pages/timeline.vue, ui/deck/tl-column.vue.
From packages/backend/src/server/api/endpoints/notes/renotes.ts, where it lists quotes, into quotes.ts beside it.
From packages/frontend/src/ui/_common_/navbar.vue into navbar-for-mobile.vue beside it.
From packages/backend/src/queue/processors/InboxProcessorService.ts into packages/backend/src/core/UpdateInstanceQueue.ts, wherever updateInstanceQueue is affected.
From .config/example.yml into .config/ci.yml.
In packages/backend/src/core/MfmService.ts, from toHtml into toMastoApiHtml.
From verifyLink in Misskey's packages/backend/src/server/api/endpoints/i/update.ts into verifyFieldLinks in packages/backend/src/misc/verify-field-link.ts, which VerifyLinksProcessorService.ts runs; i/update.ts has no verifyLink here.

If the federated user data changed at all, meaning the renderPerson function in packages/backend/src/core/activitypub/ApRendererService.ts, check that the field sets in userNeedsPublishing and profileNeedsPublishing in packages/backend/src/server/api/endpoints/i/update.ts are still correct.

One change travels the other way: a header Catodon removes must not come back through a merge. X-Misskey-Hook-Secret, which sends a webhook's secret itself, is deprecated here and removed in 27.1.0, but Misskey and Sharkey still send it. From that release a merge will try to restore it in packages/backend/src/queue/processors/SystemWebhookDeliverProcessorService.ts and packages/backend/src/queue/processors/UserWebhookDeliverProcessorService.ts; keep it out, since their headers now come from packages/backend/src/misc/webhook-headers.ts.

Compare the result both ways as you go: git diff develop and git diff sharkey/develop.

Patched dependencies

patches/ holds Catodon's patches to its dependencies. A merge that moves one of them to another version, mfm-js (Sharkey's sfm-js) in packages/*/package.json in particular, stops pnpm install with ERR_PNPM_UNUSED_PATCH until the patch is either made again for the new version or dropped. The README says how to tell which, and how to prove the result. Commit.

Regeneration

pnpm build-misskey-js-with-types   # commit
pnpm run build-assets              # locales; commit

Icons

Catodon uses Phosphor icons; upstream uses Tabler. Build the frontend with the original filenames kept, then look for what came in:

rm -rf built/
NODE_ENV=development pnpm --filter=frontend --filter=frontend-embed --filter=frontend-shared build
grep -ohrP '(?<=["'\''](ti )?)(ti-(?!fw)[\w\-]+)' --exclude \*.map -- built/ | sort -u

Replace each one with its Phosphor equivalent in packages/frontend/vite.replaceIcons.ts, rebuild, and look again until nothing is left. ti-fw is a special class defined upstream: leave it alone. Commit.

Migrations

git diff develop -- packages/backend/migration/

Read every incoming migration against Catodon's own schema changes. This is where a merge does real damage: an upstream migration whose effect Catodon already applied under a different class name will look pending on every existing deployment and try to apply it a second time. See the pre-empting section of the development guide for why, and for the compatibility pattern that repairs it.

Build, test, lint

pnpm clean && pnpm build
pnpm test
pnpm --filter backend test:e2e   # needs a test database, see development.md
pnpm --filter=backend --filter=frontend-shared lint
pnpm --filter=frontend --filter=frontend-embed eslint

Fix what you can, and write down what you could not. Nothing runs these checks for you: the Forgejo Actions workflows run typechecks, installs, a CSS lint and the locale type check, but no tests, so the commands above are yours to run.

Then push and open a pull request against develop.