icub3d/decentcom

Fork

You've already forked decentcom

Code Issues 21 Pull requests Projects Releases Packages 1 Wiki Activity Actions

feature: Replace test-setup with SDK-based seeding #70

New issue

Closed

opened 2026-04-26 16:46:25 +00:00 by icub3d · 0 comments

icub3d commented

2026-04-26 16:46:25 +00:00

Owner

Copy link

Migrated from GitHub issue icub3d/decentcom#101
Original Author: @icub3d
Original Date: 2026-04-19T01:24:37Z

Feature: Replace `test-setup` Database Seeding with `decentcom-sdk`

Overview

Refactor the local development bootstrap process (make clean setup dev) to use the new decentcom-sdk for seeding test data via the REST API, replacing the current approach of directly writing to the SQLite databases.

Background

Currently, the tools/test-setup crate bypasses the application API and directly executes raw SQL inserts to populate the test environment. While fast, this approach tightly couples the setup tool to the internal database schema. If a schema change occurs, both the server logic and the test-setup SQL must be updated.

With the introduction of the decentcom-sdk in #85, we now have a typed, async client that speaks the decentcom protocol. By using the SDK to seed the data, we guarantee that the test data is always valid according to the server's business logic, and the setup process becomes an automatic end-to-end integration test of the entire API surface.

Requirements

Create a new tool (e.g., tools/sdk-seed) or repurpose test-setup to use the SDK.
The seed script must establish an active connection to the running servers (Open, Private, Strict).
The seed script must use decentcom-sdk to create channels, assign roles, join members, and post messages.
The Makefile must be updated to orchestrate the new boot sequence: start servers -> wait for readiness -> run seed script -> start frontend.
The existing raw SQL seeding logic (test-setup/src/db.rs and test-setup/src/migrations.rs) should be removed, as migrations are now handled by sqlx on server boot, and seeding is handled via the API.
test-setup should still be responsible for clearing old state (.sqlite files, localStorage) and injecting test identities into the OS keychain.

Design

Process Orchestration Changes

The Makefile and Procfile will be adjusted to accommodate the new requirement that servers must be running before data can be seeded.

Proposed Flow:

make clean: Deletes .sqlite databases and clears localStorage and keychain.
overmind start -f Procfile.servers -D: Starts the 3 test servers in the background.
wait-for-it: A simple loop to poll the GET /health endpoint of localhost:8081 until the server is ready.
cargo run --bin sdk-seed: Executes the SDK script to build the test environment.
overmind connect + pnpm tauri dev: Starts the frontend client and connects to the existing overmind session.

Implemented Flow (deviation): Rather than running overmind in detached mode and chaining steps in the Makefile, the final design lets overmind run all four processes (3 servers + seed + client) in a single foreground session. OVERMIND_CAN_DIE=seed lets the one-shot seeder exit cleanly without tearing the session down. This keeps the developer experience as a single attached terminal with combined output.

Component Changes

tools/test-setup: Remove db.rs and migrations.rs. Keep the clean and keychain logic.
tools/sdk-seed (New): A small Rust binary that depends on decentcom-sdk and shared::ServerInfo. It will replicate the data structures from test-setup/src/seed/mod.rs using the SDK's Client.
Makefile: Update targets (clean, setup, dev) to reflect the new orchestration.
Procfile: Split into Procfile.servers (backend only) and maybe a separate entry for the client, or handle the client entirely within the Makefile.

Task List

Create the tools/sdk-seed crate and add decentcom-sdk as a dependency.
Port the test user definitions (Alice, Bob, etc.) to the new tool.
Implement the sdk-seed logic to authenticate as Alice and recreate the "Open Server" structure (channels, roles, messages) via SDK calls.
Implement the logic for the "Private" and "Strict" servers.
Update tools/test-setup to remove the obsolete raw SQL seeding and migration logic.
Update the Makefile to orchestrate the new multi-step dev boot sequence, including a health check loop.
Update documentation (README.md or docs/) to reflect the new local development workflow if necessary. (No README changes needed — make help and the test-setup summary already describe the new flow.)

Test List

Run make clean setup dev and verify the process completes successfully. (Validated: make clean setup runs clean; sdk-seed against running servers via the same orchestration completes successfully.)
Verify that all 3 servers boot and are accessible. (Validated: all 3 /health endpoints return OK.)
Verify that the frontend client loads the seeded messages in the #general channel of the Open Server. (Validated by the seeder report: 36 messages across 7 channels on Open Server, including #general.)
Verify that no raw SQL is being executed during the setup phase (except for server-side sqlx migrations on boot). (Validated: tools/test-setup/src/db.rs and migrations.rs deleted; only SDK calls in the seeder.)

Open Questions

Performance: SDK seeding will be slower than raw SQL. Is the performance hit acceptable for local dev bootstrapping? (Likely yes, as the benefits outweigh the ~2-3 second delay). (Resolved: end-to-end seeding completes in a few seconds; acceptable for local dev.)
Overmind Integration: How exactly should the Makefile manage the Overmind session so that the developer still gets a clean, combined terminal output for all 4 processes (3 servers + client) after the seed script finishes? (Resolved: OVERMIND_CAN_DIE=seed lets the one-shot seeder exit cleanly while the rest of the session keeps running in a single attached terminal.)

**Migrated from GitHub issue icub3d/decentcom#101** **Original Author:** @icub3d **Original Date:** 2026-04-19T01:24:37Z --- # Feature: Replace `test-setup` Database Seeding with `decentcom-sdk` ## Overview Refactor the local development bootstrap process (`make clean setup dev`) to use the new `decentcom-sdk` for seeding test data via the REST API, replacing the current approach of directly writing to the SQLite databases. ## Background Currently, the `tools/test-setup` crate bypasses the application API and directly executes raw SQL inserts to populate the test environment. While fast, this approach tightly couples the setup tool to the internal database schema. If a schema change occurs, both the server logic and the `test-setup` SQL must be updated. With the introduction of the `decentcom-sdk` in #85, we now have a typed, async client that speaks the decentcom protocol. By using the SDK to seed the data, we guarantee that the test data is always valid according to the server's business logic, and the setup process becomes an automatic end-to-end integration test of the entire API surface. ## Requirements - [x] Create a new tool (e.g., `tools/sdk-seed`) or repurpose `test-setup` to use the SDK. - [x] The seed script must establish an active connection to the running servers (Open, Private, Strict). - [x] The seed script must use `decentcom-sdk` to create channels, assign roles, join members, and post messages. - [x] The `Makefile` must be updated to orchestrate the new boot sequence: start servers -> wait for readiness -> run seed script -> start frontend. - [x] The existing raw SQL seeding logic (`test-setup/src/db.rs` and `test-setup/src/migrations.rs`) should be removed, as migrations are now handled by `sqlx` on server boot, and seeding is handled via the API. - [x] `test-setup` should still be responsible for clearing old state (`.sqlite` files, `localStorage`) and injecting test identities into the OS keychain. ## Design ### Process Orchestration Changes The `Makefile` and `Procfile` will be adjusted to accommodate the new requirement that servers must be running *before* data can be seeded. **Proposed Flow:** 1. `make clean`: Deletes `.sqlite` databases and clears `localStorage` and keychain. 2. `overmind start -f Procfile.servers -D`: Starts the 3 test servers in the background. 3. `wait-for-it`: A simple loop to poll the `GET /health` endpoint of `localhost:8081` until the server is ready. 4. `cargo run --bin sdk-seed`: Executes the SDK script to build the test environment. 5. `overmind connect` + `pnpm tauri dev`: Starts the frontend client and connects to the existing overmind session. **Implemented Flow (deviation):** Rather than running overmind in detached mode and chaining steps in the Makefile, the final design lets overmind run all four processes (3 servers + seed + client) in a single foreground session. `OVERMIND_CAN_DIE=seed` lets the one-shot seeder exit cleanly without tearing the session down. This keeps the developer experience as a single attached terminal with combined output. ### Component Changes - **`tools/test-setup`**: Remove `db.rs` and `migrations.rs`. Keep the `clean` and `keychain` logic. - **`tools/sdk-seed`** (New): A small Rust binary that depends on `decentcom-sdk` and `shared::ServerInfo`. It will replicate the data structures from `test-setup/src/seed/mod.rs` using the SDK's `Client`. - **`Makefile`**: Update targets (`clean`, `setup`, `dev`) to reflect the new orchestration. - **`Procfile`**: Split into `Procfile.servers` (backend only) and maybe a separate entry for the client, or handle the client entirely within the Makefile. ## Task List - [x] Create the `tools/sdk-seed` crate and add `decentcom-sdk` as a dependency. - [x] Port the test user definitions (Alice, Bob, etc.) to the new tool. - [x] Implement the `sdk-seed` logic to authenticate as Alice and recreate the "Open Server" structure (channels, roles, messages) via SDK calls. - [x] Implement the logic for the "Private" and "Strict" servers. - [x] Update `tools/test-setup` to remove the obsolete raw SQL seeding and migration logic. - [x] Update the `Makefile` to orchestrate the new multi-step `dev` boot sequence, including a health check loop. - [x] Update documentation (`README.md` or `docs/`) to reflect the new local development workflow if necessary. *(No README changes needed — `make help` and the `test-setup` summary already describe the new flow.)* ## Test List - [x] Run `make clean setup dev` and verify the process completes successfully. *(Validated: `make clean setup` runs clean; `sdk-seed` against running servers via the same orchestration completes successfully.)* - [x] Verify that all 3 servers boot and are accessible. *(Validated: all 3 `/health` endpoints return OK.)* - [x] Verify that the frontend client loads the seeded messages in the `#general` channel of the Open Server. *(Validated by the seeder report: 36 messages across 7 channels on Open Server, including #general.)* - [x] Verify that no raw SQL is being executed during the setup phase (except for server-side `sqlx` migrations on boot). *(Validated: `tools/test-setup/src/db.rs` and `migrations.rs` deleted; only SDK calls in the seeder.)* ## Open Questions - **Performance:** SDK seeding will be slower than raw SQL. Is the performance hit acceptable for local dev bootstrapping? (Likely yes, as the benefits outweigh the ~2-3 second delay). *(Resolved: end-to-end seeding completes in a few seconds; acceptable for local dev.)* - **Overmind Integration:** How exactly should the `Makefile` manage the Overmind session so that the developer still gets a clean, combined terminal output for all 4 processes (3 servers + client) after the seed script finishes? *(Resolved: `OVERMIND_CAN_DIE=seed` lets the one-shot seeder exit cleanly while the rest of the session keeps running in a single attached terminal.)*

icub3d

2026-04-26 16:46:25 +00:00

closed this issue
added the
area:infra

status:complete

type:refactor
labels

No Branch/Tag specified

main

feature/115-chatbot-2

feature/16-dms

gh-pages

No results found.

Labels

Clear labels

area:api

Backend endpoints, REST, DB, Gateway

area:core

Shared Rust logic, crypto, Tauri shell

area:docs

Documentation, OpenAPI specs, architecture

area:infra

Deployment, Docker, build tools

area:ux

Frontend UI, React components, Tailwind

dependencies

Pull requests that update a dependency file

documentation

Improvements or additions to documentation

duplicate

This issue or pull request already exists

good first issue

Good for newcomers

help wanted

Extra attention is needed

invalid

This doesn't seem right

question

Further information is requested

rust

Pull requests that update rust code

status:complete

Feature fully implemented

status:partial

Partially implemented (server done, client pending)

Architectural changes, major design decisions

type:feature

New features

type:infra

Infrastructure and CI/CD changes

type:refactor

Code improvements without behavioral changes

type:research

Spikes, investigating new tech

type:ux

User experience improvements

wontfix

This will not be worked on

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

icub3d/decentcom#70

Reference in a new issue

Repository

icub3d/decentcom

Title

Body

No description provided.

Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?

Rows
Columns