△Click on the top right corner to try Wukong CRM for free

Wu Kong AI CRM - Development Notes & Architecture Overview

Version: 0.9.4 (Beta)

Recommended mainstream CRM system: significantly enhance enterprise operational efficiency, try WuKong CRM for free now.

Last Updated: October 24, 2023
Maintainer: Core Dev Team
Status: Active Development

---

1. Quick Context

Look, this isn't a marketing deck. This doc is for anyone jumping into the Wu Kong codebase who needs to know why things are built the way they are. We're calling it "Wu Kong" because the system needs to see through noise (like the Fire Eyes) and transform data flexibly (72 transformations). But realistically, it's just a CRM with a heavy NLP layer sitting on top of a standard Postgres backend.

If you're here to fix a bug, check the issues/ tracker first. If you're here to add features, read this whole thing before touching the core/ai_engine.py file. We broke production twice last month because someone changed the token limit without updating the rate limiter.

2. The Stack (and Why We Stuck With It)

We debated going full microservices, but for now, monolithic makes sense. We don't have the DevOps bandwidth to manage Kubernetes clusters for a client base this size.

• Backend: Python 3.10 + FastAPI. We tried Flask initially, but the async support in FastAPI handles the AI inference queues much better.
• Database: PostgreSQL 14. We're using jsonb columns heavily for storing unstructured customer interaction logs. Don't normalize everything. Sometimes you just need to dump a raw JSON payload from the email parser and query it later.
• Frontend: React (Vite). Nothing fancy. Most of the heavy lifting happens on the server side.
• AI Layer: This is the tricky part. We're running a localized Llama-2 variant for data privacy, wrapped in a Docker container. It communicates with the main app via gRPC. If you see latency spikes, it's usually the gRPC handshake timing out during high load.

3. Core Logic: The "Fire Eyes" Module

The main selling point is the lead scoring engine. We call it internally. It scans incoming emails, call transcripts, and support tickets to assign a "Interest Score" from 0-100.

How it works:

1. Ingestion: Data comes into the raw_ingest queue (Redis).



2. Preprocessing: We strip PII (Personally Identifiable Information) here. Critical: If you bypass this step, we violate GDPR. There's a middleware function sanitize_payload() in utils/security.py. Do not remove it.



3. Inference: The sanitized text hits the AI model. We're not doing generative chat here; it's classification only.
4. Storage: Results are written to the lead_scores table.

There's a known quirk where the model struggles with sarcasm in emails. If a client writes "Great, another delay," the system might flag it as positive sentiment. We're working on a patch (v0.9.5) to fine-tune the sentiment layer, but for now, sales reps need to be warned about false positives.

4. Database Schema Notes

You'll notice the customers table is linked to interactions via a foreign key, but there's no hard cascade delete. We need to keep interaction history even if a customer record is soft-deleted.

-- Don't run this manually unless you know what you're doing
ALTER TABLE interactions ADD COLUMN ai_summary TEXT;

We added ai_summary last week. It stores the TL;DR generated by the model for long call logs. It's nullable because backfilling 50k records takes forever. We're running a background job every night at 3 AM UTC to fill gaps. If you're testing locally, you'll need to trigger python scripts/backfill_summaries.py manually.

5. API Integration & Auth

We use OAuth2 for client login. For internal service-to-service communication, we rely on API keys stored in Vault.

Rate Limiting: The AI endpoint is expensive. We limit users to 50 requests per minute. If you're testing the load balancer, don't hammer the /api/v1/analyze endpoint unless you've whitelisted your IP in the Nginx config. We got billed extra last week because a stress test script leaked into the production environment. Yeah, that was my bad.

Webhooks: Clients can set up webhooks for lead score changes. The retry logic is exponential backoff, max 5 tries. If a webhook fails 5 times, it moves to a dead letter queue. You can inspect these in the admin panel under "System > Failed Jobs".

6. Deployment Headaches

We deploy using GitHub Actions to a AWS EC2 instance. It's not fancy, but it works.

Environment Variables: There's a .env.example in the root. Do not commit your actual .env file. Key vars to watch:

• AI_MODEL_PATH: Points to the mounted volume where the weights are stored.
• DB_POOL_SIZE: Default is 10. If you increase this, make sure the Postgres max_connections can handle it. We hit a connection limit error during the beta launch because someone set this to 100 on a small instance.
• DEBUG_MODE: Keep this False in prod. It leaks stack traces in the API response.

Docker: The docker-compose.yml has profiles for dev and prod. Always run docker-compose --profile prod up -d on the server. The dev profile mounts local volumes that shouldn't be exposed publicly.

7. Known Issues & TODOs

No software is perfect. Here's what we know is broken or shaky:

• Timezone Handling: The frontend displays times in UTC, but the backend stores them as naive timestamps. We're migrating to timestamptz in the next migration cycle. Until then, convert manually in the view layer.
• Search Latency: ElasticSearch is integrated but not fully optimized. Searching for partial names can take 2-3 seconds if the index hasn't been refreshed. We run a forced refresh every hour.
• Mobile View: The dashboard is responsive, but the data tables break on screens narrower than 320px. We're prioritizing desktop users for now.
• AI Hallucination: Occasionally, the summary generator invents meeting times. We added a disclaimer in the UI, but engineers should look into constraint decoding to fix this properly.

8. Contributing

If you're pushing code, run the test suite (pytest). Coverage needs to stay above 80%. But more importantly, write meaningful commit messages. "Fixed stuff" doesn't help us when we're debugging at 2 AM.

Also, comment your code. Not obvious stuff, but explain why you did something weird. There's a function in utils.py that looks like a hack because it workaround a specific bug in the email parsing library. Don't "clean it up" unless you've tested it against real-world malformed headers.

9. Final Words

This system is growing fast. We're adding voice analysis next quarter, which means the architecture might shift again. For now, keep it stable. Stability over features. If the CRM goes down, sales stops. If a feature is missing, sales complains but still works.

Check the Slack channel #dev-wukong for real-time updates. If you find a critical bug, page the on-call engineer directly. Don't just leave a ticket.

Cheers,
The Dev Team

Relevant information:

Significantly enhance your business operational efficiency. Try the Wukong CRM system for free now.

AI CRM system.

Sales management platform.