After bringing Fast Sub to its current stage, I finally feel able to answer a question I kept asking myself a month ago: if I really want to use tools like Codex or Claude Code to build a complete project, how would I actually do it?
Back then, I wanted to find an article or video that showed a real project built from zero to something releasable with AI coding tools. Not a ten-minute demo, not a “type one sentence and generate an app” showcase, but a slightly complex real project: changing requirements, architecture tradeoffs, refactoring, testing, UI, packaging, QA, and cleanup before open sourcing. I looked around and found very little that I could actually reference.
To make the rest easier to understand, let me first describe the scale of Fast Sub. It ended up being much more than a simple script. It includes a Python CLI, Go product core, Go daemon/job API, Electron desktop UI, local worker, model download flow, providers, packaging, release checks, and open source documentation.
This is the complexity estimate I asked Codex to produce from the git history:
Based on the git history, from the first commit to the current state, the time span is from 2026-04-24 to 2026-05-22, about 28 calendar days; counting both start and end dates, it is within 29 days. The number of active development days with commits is about 13. There are 71 commits in total, and the latest commit is
2026-05-22 Stabilize desktop readiness tests. More precisely, this is an MVP demo built over roughly 4 calendar weeks, with about 2 weeks of active development days. In terms of human team effort, the current output feels like 2-3 people working intensely for 3-5 weeks, or a solo founder plus AI agents pushing it forward in about one month.
The project evolution can be compressed into this line:
flowchart LR
A["Python CLI<br/>Run the core subtitle flow"] --> B["Go core / daemon<br/>Stabilize product boundaries"]
B --> C["Electron mock-first<br/>Validate UI state flow first"]
C --> D["Daemon integration<br/>Connect real local capabilities"]
D --> E["Release readiness<br/>Packaging, QA, and open source cleanup"]
After going through it myself, I now understand why this kind of experience is hard to compress into a short “tutorial.” Many articles eventually turn into distilled methodology. They look correct, but when I actually start a project, I still get stuck on what to do first, when to stop, and which parts require my own judgment.
So this article is more like a note to myself from one month ago. It is not meant to prove how powerful Codex is, nor to claim that AI can replace all development work. After this round, I simply have a more concrete feeling for what it means to move a project forward together with AI.
I am still figuring this out myself. What follows is definitely not a standard answer. More accurately, it is a set of temporary lessons I learned by building Fast Sub.
When I first started using AI to write code, it was easy to fall into an illusion: as long as I described the requirement clearly, I could hand the rest over to it.
For temporary scripts, that feeling is often true. I say I want to process a file, call an API, or generate some test data. It writes the code, I run it, fix a few small issues, and the task is done.
But Fast Sub was not that kind of project. It started as a Python CLI, then added a Go product core, daemon/job API, Electron desktop app, packaging, model downloads, real providers, QA smoke tests, and open source documentation. Once the project reached that scale, it became increasingly obvious to me that I could not treat AI coding as hands-off delegation.
It is more like a collaborator with strong execution ability and broad knowledge, but one that badly needs context and boundaries. If I tell it, “For now, only build the mock; do not connect to the real daemon,” it can execute that well. If I tell it, “This JSON schema and these exit codes must not be broken,” it will try to respect that. But if those constraints exist only in my head, or are scattered in chat history from several days ago, it will eventually forget.
I kept coming back to one sentence:
Complexity does not disappear. It only moves somewhere else.
If I do not control scope during the MVP stage, complexity moves into rework later. If I do not freeze context in documentation, complexity moves into the cost of explaining things in every new conversation. If I do not expose problems during testing and QA, complexity moves to the moment when real users try the product.
That is how I now think about Codex: it is not an automatic system that can cover everything for me. It is more like a very capable teammate. I need to give it context, boundaries, and acceptance criteria, and I also need to know when to pause and rethink.
When I say MVP here, I do not mean a formal product concept. I simply mean the smallest version that can run through the core flow.
When starting from zero, I also had the impulse to think: since AI can write code so fast, maybe I can plan the whole system from the beginning. Features, architecture, documentation, tests, UI, all done at once.
Later I found this unrealistic for a personal project. Mature teams can invest heavily in upfront design because they have people, review processes, experience, and relatively stable requirement inputs. But for personal projects, many things only become clear while building. AI can help me discuss options, but it cannot make all product judgments for me.
Fast Sub’s path basically emerged while building:
Looking back, this path was not elegant, but it fit a personal project well. Each stage answered one question: can the core capability run? Does the engineering boundary need to change? What is still missing for real user usage?
But MVP does not mean letting AI start coding immediately. This is a trap I almost fell into myself. Claude Code and Codex are both very eager to start writing code after I describe a requirement. They are almost too proactive. If I am writing a temporary script, that is fine. But for a somewhat serious project, this is exactly where things can start going out of control.
I later started by asking AI to do research first: search similar projects, analyze existing approaches, compare technology stacks, list risks, and generate an initial MVP document. During this process, I would paste in materials I found, and I would also ask another model to review it from a different angle. For example, I might ask it to act like a senior architect and check whether the MVP is overdesigned; or the opposite, ask it to identify places where the plan is too optimistic.
There is nothing magical about the prompt here. At least as of 2026-05-22, my feeling is that instead of worshiping a particular incantation, it is better to clearly explain the requirement, constraints, reference projects, and my own questions. For dependency versions, GitHub Actions, Node configuration, and similar details, I later tended to ask AI to verify them online. Otherwise it can occasionally produce outdated answers, which later turn into strange warnings or build failures.
After the MVP document is ready, I ask AI to act as a project manager and split the project into several rounds. Each round states its goal, scope, and acceptance criteria. Execution then becomes much simpler: refine the round document, create a branch, implement, and finally validate according to the document.
This step looks slow, but it saves time later.
When coding with AI, it is easy to treat the prompt as the requirement document. But in complex projects, prompts are too lightweight.
A prompt is more like a verbal handoff. It can start a task, but it cannot carry long-term constraints. A feature may span multiple files, multiple modules, multiple conversations, or even multiple days. If all decisions live only in chat history, they will almost certainly be lost later.
In the later Electron development of Fast Sub, I basically switched to specs-driven work. Round 11 was the Electron mock-first shell. Round 12 connected the Go daemon. Round 13 focused on productization and release readiness. Each round had a corresponding document that stated what this round would do, what it would not do, which contracts it would affect, and how it would be accepted.
This matters a lot for AI, because AI is very good at “while I am here.” I ask it to fix a UI issue, and it may also adjust state management. I ask it to add a daemon API, and it may also adjust the renderer contract. Often it is not trying to misbehave. It is simply judging from local context that “this is more reasonable.” But in a project, not everything that is locally reasonable can be changed: CLI arguments, JSON schemas, exit codes, provider contracts, and remote upload confirmation flows are examples.
To AI, code is just code. To the project, some code is actually a contract. Once it changes, users, tests, UI, and documentation may all break. This is especially true when working on boundaries like a CLI, daemon API, or provider. I cannot let AI treat “this looks more reasonable” as “it is okay to change public behavior.”
So in a SPEC, what I later cared about most was not only “what to do,” but “what not to do.”
For example, Round 11 explicitly did not connect to the real daemon and did not call real ffmpeg, the Python worker, or provider runtimes. Round 12 was when the real daemon was connected. Round 13 focused on packaging, diagnostics, tests, and release quality, without introducing new core business architecture.
With these boundaries, AI’s freedom actually becomes more stable. It is more likely to understand that this round should only solve this round’s problems. When it finds something out of scope, it records a follow-up instead of immediately changing it.
My current habit is: before a large implementation, write a SPEC, review it myself, then ask another conversation or model to review it. The goal is not to make the document beautiful. The goal is to expose ambiguity before touching code.
The first few rounds of Fast Sub were mostly driven by conversations. At the beginning, this was fine. The project was still small, and AI could keep up. But as development continued, the problems became obvious.
The most typical case was that the project manager conversation forgot its own role and started editing code. Some conversations forgot what had already been completed, or did not know which contracts were untouchable. If I reminded it once, it would return to normal; after a while, it might drift again.
This is not because one specific model is especially bad. LLM conversations are simply not stable memory. Tools like Claude Code and Codex help me maintain context, but context is still limited. Long conversations get compressed, details are lost, and lots of no-longer-important information gets mixed in.
Later I referred to JS Mastery’s The Six-File Context System, as well as the corresponding Six-File Context System Guide download page. I did not copy it exactly. Instead, I adapted it into a set of documents that fit Fast Sub at the time. For the UI stage, the most important ones were:
project-overview.md: project goals, scope, and stage.architecture.md: architecture boundaries, module responsibilities, and data flow.code-standards.md: code style and implementation conventions.ui-context.md: UI tokens, visual rules, and component conventions.ai-workflow-rules.md: AI workflow rules.project-tracker.md: current progress, decisions, and next steps.These files were not there to make the repository look more “professional.” They were there so a new conversation could pick up the project. Before an implementation conversation starts, I ask it to read these context files. After code changes are done, the tracker is updated. Project memory no longer depends entirely on a single chat window.
The context structure I ended up with looks roughly like this:
flowchart TD
A["AGENTS.md<br/>Global working rules"] --> B["Long-term context documents"]
B --> B1["project-overview<br/>Goals and scope"]
B --> B2["architecture<br/>Architecture and boundaries"]
B --> B3["code-standards / ui-context<br/>Code and UI conventions"]
B --> B4["ai-workflow-rules<br/>AI collaboration rules"]
B --> C["Round SPEC<br/>What this stage does / does not do"]
C --> D["Implementation conversation"]
D --> E["project-tracker<br/>Results, decisions, next steps"]
E --> C
This change helped a lot later. With these files in place, Codex’s plans became noticeably closer to the actual project state, and it forgot constraints less often. The cost was that it spent a bit longer thinking each time, and the answers felt less lightweight. For a complex project, I am willing to pay that cost.
I also realized that context management does not mean stuffing everything into AI. Too little context causes misunderstanding. Too much context slows it down and may make it miss the point. For me, the smoother approach is to put long-term stable information in project documents, and current-stage information in SPECs and the tracker.
I now rarely put a complex project into a single chat window.
The longer a single conversation gets, the more it turns into a stew. Planning, implementation, review, QA, and release documentation all mixed together make it difficult for AI to maintain a stable role. I might ask it to finish an implementation round, then review its own code, then plan the next round. The contexts easily get tangled.
Later, I kept several standing conversations:
Actual implementation happens in a new conversation. The implementation conversation receives only the SPEC and context needed for the current round, without carrying too much historical baggage.
At first this felt slightly cumbersome, but later it became much more comfortable. The PM conversation acts more like a project state machine, responsible for clarifying work. The implementation conversation is more like a temporary worker that takes a clear task and executes. The review conversation focuses on finding problems. Handoffs between conversations do not rely on “remember this?” but on documents and the tracker.
For example: the PM conversation first generates the Round 12 daemon integration spec; the Plan conversation reviews it, focusing on daemon API, secrets, SSE, and configuration write boundaries; the implementation conversation connects the real daemon client according to the spec; the Review conversation checks contracts, privacy, and test gaps; finally, the QA conversation organizes issues found during real desktop testing and writes results back to the tracker. Each conversation carries one type of cognitive load.
Drawn as a loop, it looks roughly like this:
flowchart LR
A["PM conversation<br/>Split rounds / write initial spec"] --> B["Plan conversation<br/>Review spec / fill boundaries"]
B --> C["Implementation conversation<br/>Change code according to spec"]
C --> D["Review conversation<br/>Check contracts / tests / privacy"]
D --> E["QA conversation<br/>Organize real issues"]
E --> F["Tracker<br/>Persist state and next steps"]
F --> A
Some rounds can be split further into multiple worktrees and developed in parallel. I once tried up to 4 worktrees at the same time, and the efficiency was genuinely high. But there is one precondition: the tasks need to be cut cleanly enough. If several branches all modify the same state management or the same contract, merging becomes painful.
My workflow is basically to keep the main branch clean, rebase feature branches onto the primary branch (main / master), then merge with fast-forward. This makes review and rollback clearer. This may not fit everyone, but for a personal project, it was easier to control than having many branches merge into each other.
If each conversation is also given some dedicated skills, it starts to feel a bit like agents. But I do not want to make it sound mystical. For code development, simply making different conversations responsible for different roles already solves many problems.
After using Codex, it became hard for me to review every generated line one by one. Not because I did not want to, but because it was not realistic. AI generates code too fast. Once the project becomes complex, human line-by-line review quickly falls behind.
My feeling later was that tests cannot be decorative. They have to participate in deciding whether a change broke something.
In Fast Sub, many things cannot be changed casually: CLI command names, arguments, JSON schemas, exit codes, provider contracts, daemon APIs, secret redaction, and remote upload confirmation. Verbal reminders are not enough. I need to write these into documentation and cover them by tests as much as possible.
The validation I ran differed by stage. On the Python side, there were ruff, mypy, and pytest. On the Go side, there was go test. On the Electron side, there were typecheck, unit tests, build, and smoke tests. By Round 13, I also needed packaged smoke, installer smoke, real local provider/file smoke, long-task cancellation smoke, screenshot baselines, and license inventory.
But automated tests are not everything. I still need to manually run the core flow. This is especially true for desktop apps. Many problems only appear when I actually click through the app: whether a button is clickable, whether a long filename breaks a dialog, whether error messages are understandable, whether task state jumps after switching away and back, whether GPU processes remain after canceling a job.
I also hit issues with refactoring.
After the Python part was done, the project already had more than ten Python files, and some files were thousands of lines long. That was when I realized I was paying debt for not defining the project architecture and code style earlier. If there had been clearer context files and code boundaries from the beginning, the later pain might have been smaller.
At first I wanted Codex to refactor everything in one shot based on a reasonable architecture diagram. The result was not ideal. It could produce a directory structure that looked good, but at the concrete file level it often used shims to route around the problem, and the code was not really moved much.
In the end, I had to point out issues one directory at a time and let Codex make smaller changes. Fortunately, the test coverage was good enough that after each change I could quickly verify whether behavior had been broken.
My view on AI refactoring has become more conservative since then: it is very good at splitting files, extracting types, and organizing modules, but only if I first define what “not broken” means. Without tests, boundaries, and a small-step rhythm, refactoring can easily become another disaster.
After the Python and Go parts were completed, UI was what worried me most, because I had almost no experience building a complete desktop UI.
The first time I used Claude Design to generate prototypes, I was honestly stunned. I only gave it a rough page description and the daemon-api.md iterated earlier, and it produced several visual styles plus prototypes for more than a dozen major pages. It felt like I was still describing requirements with stone-age tools, while it had already placed an entire interface world in front of me.
But the shock soon turned into another practical problem: this kind of visual iteration consumes a lot of quota and context. After changing only a few pages, the account quota would already start to feel tight.
Later I downloaded the prototype files and let Codex continue modifying them. One detail I only realized later: prototype code alone is not enough. If Codex only sees the code, it is hard for it to reliably reproduce the visual effect. I later provided screenshots of each prototype page as well, so it could reference both structure and final appearance.
Looking back, there may already be easier ways to do this now. For example, Open Design can basically be understood as an open source alternative to Claude Design. It connects the design-generation workflow to existing coding-agent CLIs, including Codex, Claude Code, Cursor, Gemini, OpenCode, and others. If I had used something like that at the time, I might not have needed to move prototype files, screenshots, and revision notes back and forth between Claude Design and Codex.
Once the prototype was roughly ready, I did not connect the real backend immediately. I went mock-first.
Fast Sub’s Round 11 was the Electron mock-first shell: establish the FastSubClient contract, Mock client, first launch flow, main screen, job queue, and settings page. This stage did not connect to the real daemon, and did not call real ffmpeg, Python workers, or model downloads.
This later proved very worthwhile. The UI could validate information architecture and state flow first, without being blocked by backend readiness. When Round 12 connected the real daemon, the client contract already existed, and the pages did not need to be rebuilt.
If the UI had connected to the real daemon from the beginning, problems would have been mixed together: if a button did not respond, was it a UI state bug, a daemon API bug, or a job event mapping bug? Mock-first removed at least half of that uncertainty.
So if I build a similar project again, I will probably still mock first. Even if part of the backend is already available, I would rather smooth out the user flow with mocks first, then gradually replace them with real implementation.
After connecting the real daemon worker, the project entered the stage where I spent the most time.
When building command-line tools, I had always wondered: why does wrapping a CLI in a UI shell noticeably lower the barrier to use? AI is so convenient now, so why are many tools still stuck at the command line?
After actually building a desktop version myself, I understood. The last mile of UI is extremely fragmented, and many problems are hard to catch ahead of time with automated tests.
For example: is the button where users expect it to be? Will very long batch filenames break the confirmation dialog? Is the error message understandable after a task fails? Does the state jump when a generating task goes to the background and comes back? Are GPU processes left behind after canceling a long task? These are not problems I can confidently solve with one unit test.
I spent more than a week on this part. Honestly, it was quite draining.
The thing that improved efficiency a bit was building a QA test table. I stopped opening a new conversation and fixing one issue immediately every time I found a problem. Instead, I recorded issues in batches, classified them in batches, and then let Codex handle them by category. This is much more efficient, and it also makes it easier to confirm which issues have been fixed and which still need retesting.
Late-stage Fast Sub QA covered many scenarios that only fail in real usage:
There is another point I only truly understood later: the packaged app is the real product. Dev mode running successfully only proves that it runs in the development environment. In a packaged app, many previously invisible issues appear: app-private Python runtime, daemon cwd, resource paths, Windows installer, portable zip, child processes left after exit, macOS signing and permissions. Fast Sub Round 13 spent a lot of time on these things, and looking back, it was worth it, because they determine whether a user can actually use the app after downloading it.
Before open sourcing, there was another category of cleanup that did not look like coding, but was not small at all: README, CHANGELOG, CONTRIBUTING, SECURITY, LICENSE, third-party dependency license inventory, privacy notes, installation instructions, and troubleshooting documentation.
I also had to clean up things that cannot be public, such as API keys, local machine paths, large model files, large media files, real benchmark output, and temporary build artifacts.
I underestimated this step at first. For developers, working code can feel like the end. But for external users, documentation, privacy notes, installation instructions, and known limitations are all part of the project’s credibility. AI is good at generating a first draft of documentation, but I still have to review the promises in those docs myself. Installation instructions, privacy notes, and known limitations in particular cannot be allowed to sound too optimistic.
Fast Sub is also local-first, so the privacy boundary has to be explicit. Remote providers must not become implicit behavior. I want any path that uploads audio or subtitle text to be explicitly selected and confirmed by the user.
If I compress this experience into a few pitfalls, they would be these.
At first, I trusted AI too much to “write and organize as it goes.” The result was that the Python code later had large files and mixed responsibilities, and I had to spend dedicated time refactoring.
Early context depended too much on chat windows. Once a conversation became long, AI easily forgot constraints and sometimes even started doing things it should not do.
At some stages I asked AI to do too much at once. When the scope became large, bugs and drift stacked together, making later debugging exhausting.
The early UI prototypes were not detailed enough. Many interaction issues only appeared during real QA.
I also underestimated the difference between packaged apps and real environments. Running in dev mode does not mean running after packaging. System Python, app-private Python, daemon cwd, process cleanup, and path permissions are only truly exposed after packaging.
Behind all these pitfalls is the same issue: complexity does not disappear, it only moves.
AI can help me write code faster, but it cannot make complexity vanish. Problems I do not handle early will still appear later during implementation, QA, or real usage.
If I were to build Fast Sub again from scratch now, I would not overturn the overall path, but there are several things I would do earlier.
First, create context files from the beginning. I would not wait until conversations start forgetting things and code starts swelling. Even rough project-overview, architecture, code-standards, and project-tracker files are better than relying entirely on chat history.
Second, define directory structure and code style earlier. Refactoring after early Python files grew large cost more than I expected. AI can help me refactor, but it is not good at absorbing the consequences of “we did not define boundaries earlier” on my behalf.
Third, during the UI prototype stage, organize screenshots, states, and copy more carefully. The rougher the prototype, the more interaction detail I need to patch during QA. Especially for desktop tools, I would try to think through empty states, error states, batch jobs, cancellation, and failed retry flows during the mock stage.
Fourth, establish a QA table earlier. I do not want to wait until last-mile issues explode before systematically recording them. A QA table is not only a test checklist; it is also an input format for collaborating with AI on bug fixes.
Fifth, separate feature work, refactoring, and release work more strictly in every round. AI easily mixes them together, but their acceptance criteria are completely different. For features, I check whether user capability increased. For refactoring, I check whether behavior stayed the same. For release work, I check whether the real environment runs successfully.
None of these are flashy tricks, but if I had done them earlier, I probably would have carried less debt.
After building this project with Codex, my view of AI coding changed quite a bit.
I no longer think the key is finding a universal prompt. Prompts are useful, skills are useful, and tools will keep getting stronger. But after this project, I feel more clearly that the parts that really consume energy in complex projects are scope control, context, validation, and closing things out.
If I were to turn this experience into a note for myself, I would write down these things:
None of this sounds cool, and it is less attractive than a single magical prompt. But these are the things that stayed with me after this project.
AI coding is not something I can hand off and forget. It is more like bringing a very capable collaborator into the development process. The clearer the process is, the more it amplifies human ability. The messier the process is, the faster it amplifies the mess.
That is probably what I most want to tell myself from one month ago after finishing this project.
]]>Fast Sub 做到现在这个阶段后，我发现自己终于能回答一个月前一直想问的问题了：如果真的想用 Codex、Claude Code 这类工具做一个完整项目，到底应该怎么做？
我当时很想找一篇真正从 0 开始，用 AI 编程工具把项目做到能发布的文章或者视频。不是那种十分钟 demo，也不是“输入一句话生成一个 App”的展示，而是一个稍微复杂一点的真实项目：会有需求变化、架构取舍、重构、测试、UI、打包、QA、开源前整理。找了一圈之后，能参考的东西不多。
先交代一下 Fast Sub 的体量，方便理解后面为什么我一直在讲上下文、测试和 QA。它最后不是一个单纯脚本，而是包含 Python CLI、Go product core、Go daemon/job API、Electron 桌面 UI、本地 worker、模型下载、provider、打包、发布检查和开源文档的一整套项目。
下面这段是我让 Codex 根据 git 历史给出的复杂度估算：
按 git 历史看，从 first commit 到当前状态，时间跨度是 2026-04-24 到 2026-05-22，约 28 天自然日；如果按包含首尾日期说，就是 29 天内。真正有提交的活跃开发日大概是 13 天，总提交数 71 个，最新提交是
2026-05-22 Stabilize desktop readiness tests。更准确地说，它是一个约 4 周自然时间、集中开发大概 2 周左右活跃工作日推进出来的 MVP Demo。按人类小队估算，当前产出更像是 2-3 人高强度干 3-5 周，或者一个 solo founder 加 AI agent 在约 1 个月里推出来的结果。
项目演进大概可以压成这条线：
flowchart LR
A["Python CLI<br/>跑通核心字幕流程"] --> B["Go core / daemon<br/>稳定产品边界"]
B --> C["Electron mock-first<br/>先跑通 UI 状态流"]
C --> D["Daemon integration<br/>接入真实本机能力"]
D --> E["Release readiness<br/>打包、QA、开源前收口"]
现在自己做完一遍，大概也知道为什么了。因为这种体量的经验，很难被压缩成一个几千字的“教程”。很多文章最后都会变成提纯过的方法论，看起来很正确，但我真正开项目时，还是会卡在第一步该怎么做、什么时候该停下来、哪些地方要自己判断这些问题上。
所以这篇文章更像是给一个月前的我写的备忘录。它不是要证明 Codex 多强，也不是要说 AI 可以替我完成所有开发。只是做完这一轮后，我对“怎么和 AI 一起推进一个项目”有了一些更具体的感受。
我自己也还在摸索，下面的内容肯定不是标准答案。更准确地说，它只是我用 Fast Sub 这个项目踩出来的一套阶段性经验。
刚开始用 AI 写代码的时候，很容易有一种错觉：只要把需求说清楚，后面就可以交给它自己推进。
写临时脚本时，这个感觉确实成立。我说要处理一个文件、调一个 API、生成一段测试数据，它写出来，我跑一下，修几个小问题，事情就结束了。
但 Fast Sub 不是这种项目。它从 Python CLI 开始，后来加了 Go product core、daemon/job API，再到 Electron 桌面端、打包、模型下载、真实 provider、QA smoke 和开源文档。项目一旦到这个规模，我越来越明显地感觉到，不能把 AI 编程当成放手托管。
它更像一个执行力很强、知识面很广、但非常需要上下文和边界的协作者。我告诉它“现在只做 mock，不要接真实 daemon”，它可以很好地执行。我告诉它“这个 JSON schema 和退出码不能破坏”，它也能尽量遵守。但如果这些约束只存在我脑子里，或者散落在前几天的聊天记录里，那它迟早会忘。
我后来反复想到一句话：
复杂度不会消失，只会转移。
我不在 MVP 阶段控制范围，复杂度就会转移到后面的返工里。我不在文档里固定上下文，复杂度就会转移到每个新对话的解释成本里。我不在测试和 QA 阶段暴露问题，复杂度就会转移到用户真正使用的时候。
这也是我现在看 Codex 的方式：它不是一个可以替我兜底的全自动系统，更像是一个很强的队友。我得给它上下文、边界、验收标准，也得知道什么时候该暂停下来重新判断。
我这里说的 MVP，不是一个很正式的产品概念，只是第一个能跑通核心流程的最小版本。
从零做项目时，我一开始也会有冲动：既然 AI 写代码这么快，那是不是可以一上来就把完整系统规划好？功能、架构、文档、测试、UI，全都一次性做出来。
后来发现，这个想法对个人项目不太现实。成熟团队可以在前期做大量设计，因为他们有人力、有评审、有经验，也有比较稳定的需求输入。但个人项目很多时候是在做的过程中才逐渐想清楚。AI 可以帮我讨论方案，但不能替我做所有产品判断。
Fast Sub 的路线基本是边做边收敛出来的：
现在回头看，这条路线不算优雅，但它很适合个人项目。每一阶段都能回答一个问题：核心能力能不能跑？工程边界要不要调整？真实用户使用时还缺什么？
不过 MVP 不等于让 AI 直接开写。这个坑我自己也差点踩进去。Claude Code、Codex 都很容易在我提完需求后立刻开始写代码，它们太积极了。如果只是写一个临时脚本，这没问题。但如果是一个稍微严肃一点的项目，从这里开始就可能失控。
我后来会先让 AI 做调研：搜索类似项目，分析现有方案，比较技术栈，列风险，再生成一份初版 MVP 文档。这个过程里，我会把自己看到的资料贴进去，也会让另一个模型从不同角度审一遍。比如让它假设自己是资深架构师，看看这个 MVP 有没有过度设计；或者反过来，让它挑一下哪些地方太乐观。
这里 prompt 没有多神秘。至少到 2026-05-22 这个阶段，我的感受是，与其迷信某句咒语，不如把需求、约束、参考项目和自己的疑问讲清楚。有些依赖版本、GitHub Actions、Node 配置之类的信息，我后来也会让 AI 联网确认一下。不然它偶尔会给出过时答案，后面就会变成莫名其妙的 warning 或构建失败。
MVP 文档出来后，我会继续让 AI 扮演 project manager，把项目拆成几轮 round。每轮写清楚目标、范围和验收方式。后面执行时就简单很多：先完善 round 文档，再切分支实现，最后按文档验收。
这一步看起来慢，但它是在帮后面省时间。
用 AI 编程时，很容易把 prompt 当成需求文档。但复杂项目里，prompt 太轻了。
prompt 更像一次口头交代，可以启动任务，但承载不了长期约束。一个功能可能会跨多个文件、多个模块、多个对话，甚至跨几天时间。如果所有决策都只在聊天记录里，后面基本一定会丢。
Fast Sub 后期的 Electron 开发，我基本改成 specs-driven。Round 11 做 Electron mock-first shell，Round 12 接 Go daemon，Round 13 做产品化和发布准备。每一轮都有对应文档，里面写清楚这一轮做什么、不做什么、会影响哪些 contract、完成后怎么验收。
这对 AI 特别重要。因为 AI 很擅长“顺便”。我让它修一个 UI 问题，它可能顺便改了状态管理。我让它补一个 daemon API，它可能顺便调整了 renderer contract。很多时候它不是故意乱来，只是它会根据局部上下文判断“这样更合理”。但项目里有些东西不是局部合理就能改的，比如 CLI 参数、JSON schema、退出码、provider contract、远程上传确认逻辑。
对 AI 来说，代码只是代码；但对项目来说，有些代码其实是 contract。它们一旦变了，用户、测试、UI、文档都会跟着坏。尤其是做 CLI、daemon API、provider 这类边界时，不能让 AI 把“看起来更合理”当成“可以改公开行为”。
所以 SPEC 里我后来最看重的，其实不是“要做什么”，而是“不能做什么”。
比如 Round 11 明确不接真实 daemon，不调用真实 ffmpeg、Python worker 或 provider runtime。Round 12 才接真实 daemon。Round 13 聚焦打包、诊断、测试和发布质量，不再引入新的核心业务架构。
有了这些边界，AI 的发挥空间反而更稳定。它更容易知道这一轮只该解决这一轮的问题，遇到超出范围的东西就记录 follow-up，而不是直接动手。
我现在的习惯是：大的实现前先写 SPEC，自己审一遍，再让另一个对话或模型审一遍。不是为了写得多漂亮，而是为了在动代码前先把模糊的地方暴露出来。
Fast Sub 前几轮基本是靠对话推进的。一开始没什么问题，项目还小，AI 也能跟得上。但后面继续做时，问题开始变明显。
最典型的情况是 project manager 对话忘记了自己的角色，开始改代码。还有些对话会忘记之前完成了什么，或者不知道哪些 contract 是不能动的。我提醒它一次，它会恢复正常；过一段时间，又可能偏掉。
这不是某个模型特别差，而是 LLM 对话本来就不是稳定记忆。Claude Code、Codex 这些工具会帮我维护上下文，但上下文终究有限。对话长了会压缩，会丢细节，也会混入很多已经不重要的信息。
后来我参考了 JS Mastery 的 The Six-File Context System 这套做法，也看了它对应的 Six-File Context System Guide 下载页。我没有完全照搬，而是按 Fast Sub 当时的状态改成了一组更适合自己项目的文档。以 UI 阶段为例，比较核心的是这几个：
project-overview.md：项目目标、范围和阶段。architecture.md：架构边界、模块职责和数据流。code-standards.md：代码风格和实现约定。ui-context.md：UI token、视觉规则和组件约定。ai-workflow-rules.md：AI 工作流规则。project-tracker.md：当前进度、决策和下一步。这些文件不是为了让仓库看起来更“工程化”，而是为了让新对话能接得上。实现对话开始前，我会让它先读这些上下文。改完代码后，再更新 tracker。这样项目记忆就不再完全依赖某一个聊天窗口。
我后来理解的上下文结构，大概是这样：
flowchart TD
A["AGENTS.md<br/>全局工作规则"] --> B["长期上下文文档"]
B --> B1["project-overview<br/>目标和范围"]
B --> B2["architecture<br/>架构和边界"]
B --> B3["code-standards / ui-context<br/>代码和 UI 约定"]
B --> B4["ai-workflow-rules<br/>AI 协作规则"]
B --> C["Round SPEC<br/>当前阶段要做什么 / 不做什么"]
C --> D["实现对话"]
D --> E["project-tracker<br/>结果、决策、下一步"]
E --> C
这个改变对后续帮助很大。有了这些文件后，Codex 给出的方案明显更贴近项目现状，也更少出现忘记约束的情况。代价是它每次思考会更久一点，回答也没那么轻快。但对复杂项目来说，我愿意付这个成本。
我后来也意识到，上下文管理不是把所有东西都塞给 AI。上下文太少，它会误解；上下文太多，它会变慢，也可能抓不住重点。所以对我来说更顺的方式，是把长期稳定的信息写进项目文档，把当前阶段的信息写进 SPEC 和 tracker。
我现在不太会把一个复杂项目塞进一个聊天窗口里。
单个对话越长，越容易变成一锅粥。规划、实现、review、QA、发布文档全混在一起后，AI 很难稳定保持同一个角色。我刚让它做完一轮实现，又让它审自己的代码，再让它规划下一轮，它很容易把上下文搅在一起。
我后面比较固定地保留了几个常驻对话：
具体实现则重新开对话。实现对话只拿本轮需要的 SPEC 和上下文，不背太多历史包袱。
这个做法一开始看起来有点麻烦，但后面确实舒服很多。PM 对话更像项目状态机，负责把事情拆清楚；实现对话更像临时工，拿到明确任务就做；review 对话则专门挑问题。不同对话之间不靠“还记得吗”，而靠文档和 tracker 交接。
举个具体一点的流程：PM 对话先生成 Round 12 的 daemon integration spec；Plan 对话审一遍，重点看 daemon API、secret、SSE、配置写入这些边界有没有缺口；实现对话按 spec 接入真实 daemon client；Review 对话再检查 contract、隐私、测试遗漏；最后 QA 对话整理真实桌面测试里发现的问题，再把结果回写到 tracker。这样每个对话都只承担一类认知负担。
如果画成图，大概是这个循环：
flowchart LR
A["PM 对话<br/>拆 round / 写初版 spec"] --> B["Plan 对话<br/>审 spec / 补边界"]
B --> C["实现对话<br/>按 spec 改代码"]
C --> D["Review 对话<br/>查 contract / 测试 / 隐私"]
D --> E["QA 对话<br/>整理真实问题"]
E --> F["Tracker<br/>沉淀状态和下一步"]
F --> A
有些 round 还能进一步拆成多个 worktree 并行做。我最多试过 4 个 worktree 同时开发，效率确实很高。但这里有一个前提：任务要切得足够干净。最后如果几个分支都改同一块状态管理、同一个 contract，合并时就会很痛苦。
我的工作流基本是主分支保持干净，功能分支 rebase 到主分支（main / master）后 fast-forward 合并。这样 review 和回滚都比较清楚。这个做法不一定适合所有人，但至少对我这种个人项目来说，比一堆分支互相 merge 要更容易控制。
每个对话再加上一些专属 skills，其实就有点像 agents 了。不过我现在还不想把它说得太玄。对代码开发来说，先做到“不同对话承担不同职责”，已经能解决很多问题。
用 Codex 之后，我很难再逐行 review 每一行代码。不是不想，而是不现实。AI 生成代码的速度太快，项目一复杂，靠人工一行行看，很快就会跟不上。
后来我的感受是，测试不能只是“有就行”的装饰，它得真的参与判断这次修改有没有把东西弄坏。
Fast Sub 里有很多东西不能被随便改：CLI 命令名、参数、JSON schema、退出码、provider contract、daemon API、secret redaction、远程上传确认。这些东西只靠口头提醒不够，要写进文档，也要尽量有测试覆盖。
不同阶段我跑的验证不一样。Python 侧有 ruff、mypy、pytest。Go 侧有 go test。Electron 侧有 typecheck、unit test、build、smoke。到了 Round 13，还要做 packaged smoke、installer smoke、真实本地 provider/file smoke、长任务取消 smoke、截图 baseline、license inventory。
但自动测试也不是万能的。核心流程我还是会自己手动跑一遍。尤其是桌面应用，很多问题只有真实点一遍才知道：按钮是不是能点，弹窗会不会被长文件名撑爆，失败信息用户能不能看懂，任务取消后进程有没有残留。
重构这块我也踩过坑。
Python 部分做完后，项目里已经有十几个 Python 文件，有些文件甚至几千行。那一刻我才意识到，前期没有规定好项目架构和代码风格，是会还债的。如果一开始就有更明确的 context 文件和代码边界，后面可能不会这么痛。
我一开始想让 Codex 直接根据合理架构图一步到位重构。结果并不理想。它能给出看起来不错的目录结构，但到了具体文件级别，经常用 shim 的方式把问题绕过去，代码其实没怎么挪。
最后我只能一个目录一个目录地指出问题，让 Codex 小范围改。好在测试比较齐全，每次改完都能很快验证有没有破坏行为。
后来我对 AI 重构的判断变得保守了：它很适合拆文件、抽类型、整理模块，但前提是我先定义什么叫“没改坏”。如果没有测试，没有边界，没有“小步改”的节奏，重构很容易变成另一场灾难。
Python 和 Go 这部分完成后，我最担心的是 UI。因为我几乎没有完整做过桌面 UI。
结果第一次用 Claude Design 生成原型时，我是真的被震住了。我只给了很粗糙的页面描述和前面迭代出来的 daemon-api.md，它就给出了几种风格，以及十几个主要页面的原型图。那种感觉有点像：我还在用石器时代的方式描述需求，它已经把一整个界面世界摆在我面前了。
不过震惊很快变成了另一个现实问题：这种视觉迭代非常吃额度，也非常吃上下文。基本每改几个页面，账号额度就开始紧张。
后来我把原型文件下载下来，让 Codex 继续接手修改。这里有一个细节我后来才意识到：只给原型代码不够。Codex 只看代码，很难稳定还原视觉效果。我后来会同时给每个原型页面的截图，让它同时参考结构和最终视觉。
现在回头看，这一步现在可能已经有更省事的做法了。比如 Open Design，基本可以理解成 Claude Design 的开源替代方案。它把设计生成流程接到现有的 coding-agent CLI 上，包括 Codex、Claude Code、Cursor、Gemini、OpenCode 等。如果当时已经用上这类工具，我可能就不用在 Claude Design 和 Codex 之间来回搬原型文件、截图和修改说明。
原型差不多后，我没有直接接真实后端，而是先做 mock-first。
Fast Sub 的 Round 11 就是 Electron mock-first shell：先建立 FastSubClient contract、Mock client、首次启动、主界面、任务队列、设置页。这个阶段不接真实 daemon，不调用真实 ffmpeg、Python worker 或模型下载。
这一步后来证明很值。UI 可以先把信息架构和状态流跑通，后端没完成时也不会阻塞。等到 Round 12 接真实 daemon 时，前面已经有了 client contract，页面也不需要推倒重来。
如果一开始 UI 就接真实 daemon，问题会混在一起：这个按钮没反应，到底是 UI 状态错了，还是 daemon API 错了，还是 job event 映射错了？mock-first 至少能先把一半不确定性拿掉。
所以现在让我再做类似项目，我大概率还是会先 mock。哪怕后端已经有一部分可用，我也更愿意先用 mock 把用户流程走顺，再逐步替换成真实实现。
接入真实 daemon worker 后，项目进入了我花时间最多的阶段。
做命令行工具时，我一直有点好奇：为什么只是给 CLI 套一个 UI 壳，就会明显降低使用门槛？现在 AI 这么方便，为什么很多工具还是停在命令行？
等自己真的做 desktop 版本后，我大概明白了。UI 的最后一公里非常碎，而且很多问题很难靠自动化测试提前发现。
比如按钮是不是在用户预期的位置，批量文件名太长会不会把确认弹窗撑爆，任务失败后错误信息是不是能看懂，生成中的任务切到后台再回来状态会不会跳，取消长任务后 GPU 进程有没有残留。这些问题不是写一个单元测试就能放心的。
这部分我做了一周多，说实话挺消耗耐心。
后来稍微提高效率的办法，是建立 QA 测试表。我不再发现一个问题就立刻开一个新对话修一个问题，而是批量记录、批量分类，再让 Codex 按类别处理。这样效率会高很多，也方便回头确认哪些问题已经修、哪些还要复测。
Fast Sub 后期 QA 覆盖了不少“真实使用才会出问题”的场景：
这里还有一个我后面才真正体会到的点：打包后才是真实产品。dev 模式能跑，只能说明开发环境里能跑。到了 packaged app，很多之前看不到的问题才会冒出来，比如 app 私有 Python runtime、daemon 的 cwd、资源路径、Windows installer、portable zip、退出后子进程残留、macOS 签名和权限。Fast Sub Round 13 花了很多时间在这些事情上，回头看很值，因为它们才决定一个用户下载后能不能真的用。
开源前还有另一类收口工作，看起来不像写代码，但一点也不少：README、CHANGELOG、CONTRIBUTING、SECURITY、LICENSE、第三方依赖 license inventory、隐私说明、安装说明、排障文档。
还要清理不能公开的东西，比如 API key、本机路径、大模型文件、大型媒体文件、真实 benchmark 输出、临时构建产物。
这一步我一开始也低估了。对开发者来说，代码能跑好像就结束了。但对外部用户来说，文档、隐私说明、安装说明和已知限制，都是项目可信度的一部分。AI 很适合帮我生成第一版文档，但文档里的承诺还是要自己审。尤其是安装说明、隐私说明、已知限制，不能让 AI 写得太乐观。
Fast Sub 又是本地优先工具，所以隐私边界得写清楚。远程 provider 不能变成隐式行为，任何上传音频或字幕文本的路径，我都希望它是用户显式选择和确认的。
如果把这次经历压缩成几个坑，大概是这些。
一开始太相信 AI 可以边写边整理。结果 Python 代码后面出现大文件和职责混杂，不得不专门花时间重构。
前期上下文太依赖聊天窗口。对话一长，AI 就容易忘记约束，甚至开始做自己不该做的事情。
有些阶段让 AI 一次做太多。scope 一大，bug 和偏差就会叠在一起，后面排查起来很累。
UI 原型早期不够细。后面真实 QA 时，很多交互问题才暴露出来。
还有就是低估了 packaged app 和真实环境的差异。dev 模式能跑，不代表打包后能跑。系统 Python、app 私有 Python、daemon cwd、进程清理、路径权限，这些问题只有打包后才真的暴露。
这些坑背后其实是同一个问题：复杂度不会消失，只会转移。
AI 可以帮我更快写代码，但不能让复杂度消失。有些问题早期不处理，后面还是会在实现、QA 或真实使用时冒出来。
如果现在让我从头再做一次 Fast Sub，我不会推翻现在的路线，但有几件事一定会提前做。
第一，项目一开始就建立 context 文件。不是等到对话开始忘事、代码开始膨胀后再补。哪怕一开始只是很粗的 project-overview、architecture、code-standards 和 project-tracker，也比全靠聊天记录强。
第二，更早规定目录结构和代码风格。Python 前期文件膨胀后再重构，成本比我预期高。AI 可以帮我重构，但它不擅长替我承担“前期没定边界”的后果。
第三，UI 原型阶段会把截图、状态和文案整理得更细。原型越粗，后面 QA 时补交互细节就越多。尤其是桌面工具，空状态、错误状态、批量任务、取消、失败重试这些页面，我会尽量在 mock 阶段就想清楚。
第四，更早建立 QA 表。我不想再等到最后一公里问题爆出来，才开始系统记录。QA 表不仅是测试清单，也是和 AI 协作修 bug 的输入格式。
第五，每个 round 更严格地区分 feature、refactor 和 release work。AI 很容易把它们混在一起，但这三类工作的验收标准完全不同。feature 看用户能力有没有增加，refactor 看行为有没有保持，release work 看真实环境有没有跑通。
这些都不是很炫的技巧，但如果一开始就做，我大概能少还不少债。
用 Codex 做完这个项目后，我对 AI 编程的看法变了不少。
我不再觉得关键是找到某个万能 prompt。prompt 有用，skills 有用，工具也会越来越强。但这次做下来，我更明显地感觉到，复杂项目真正消耗人的地方，还是范围控制、上下文、验证和收尾。
如果只给这次经历做一个备忘录，我大概会记下这几件事：
这些听起来都不酷，也没有一句神奇 prompt 来得吸引人。但这是我这次真正留下来的东西。
AI 编程不是把项目交出去就结束了。它更像是把一个很强的协作者接进开发流程。流程越清楚，它越能放大人的能力。流程越混乱，它也会更快地放大混乱。
这大概就是我目前做完这个项目后，最想写给一个月前自己的结论。
]]>“There is no true randomness in nature, only patterns we haven’t understood yet.” — Philosophy of Perlin Noise
Perlin Noise is a gradient noise algorithm developed by Ken Perlin in 1983 for the film Tron, and formally published in 1985 in the SIGGRAPH paper An Image Synthesizer. Perlin received an Academy Award for this contribution.
It is a type of coherent noise with these characteristics:
-1 to 1, or normalized to 0 to 1)Pure Random (White Noise): 9 4 1 7 3 8 2 5 6 ...
Perlin Noise: 3 4 5 6 5 4 3 2 3 ...
Pure random noise (white noise) treats each pixel independently, creating a “TV static” effect that almost never appears in nature.
Real natural phenomena (mountains, clouds, flames, water surfaces) exhibit local coherence: nearby locations have similar values, but with overall macroscopic variation. Perlin Noise mimics exactly this property.
| Property | White Noise | Perlin Noise |
|---|---|---|
| Smooth neighbors | ❌ | ✅ |
| Natural looking | ❌ | ✅ |
| Controllability | Low | High |
| Computational complexity | O(1) | O(2ⁿ), n = dimensions |
Perlin Noise follows three core steps:
① Create grid, assign random gradient vectors to each grid point
↓
② Compute distance vectors from sample point to surrounding points
Dot product with gradient vectors
↓
③ Use smooth interpolation to blend contributions from surrounding points
At each integer coordinate (grid point), pre-assign a random unit direction vector—this is the “gradient.”
Unlike Value Noise (storing random values at points), Perlin Noise stores directions, not values. This produces smoother, more natural output.
Simple linear interpolation (lerp) produces linear artifacts. Perlin uses a quintic smoothing curve:
Properties:
Early versions used cubic curve $3t^2 - 2t^3$ (Smoothstep); improved versions use quintic (Smootherstep).
For a 1D sample point $x = 2.3$:
Left point: x0 = floor(2.3) = 2
Right point: x1 = x0 + 1 = 3
Local coordinate: t = 2.3 - 2 = 0.3
In 1D, gradients are just two directions: +1 or -1, determined via permutation table:
grad(2) = +1
grad(3) = -1
Distance vector × gradient vector (multiplication in 1D):
Left contribution: dot(grad(x0), x - x0) = (+1) × (2.3 - 2) = +0.3
Right contribution: dot(grad(x1), x - x1) = (-1) × (2.3 - 3) = +0.7
u = fade(t) = fade(0.3) ≈ 0.163 (quintic curve)
result = lerp(u, 0.3, 0.7) = 0.3 + 0.163 × (0.7 - 0.3) ≈ 0.365
In 2D, sample point $(x, y)$ is surrounded by four grid points:
(x0,y1) -------- (x1,y1)
| . |
| (x,y) |
| |
(x0,y0) -------- (x1,y0)
2D gradients typically chosen from 4 (or 8) vectors:
(1,0), (-1,0), (0,1), (0,-1)
(1,1), (-1,1), (1,-1), (-1,-1) ← normalized version
① Calculate distance vectors from sample to 4 corners
② Dot each with corner gradient → 4 values
③ Interpolate along x using fade(tx), then along y with fade(ty)
# Pseudocode
a = dot(grad(x0, y0), dx, dy )
b = dot(grad(x1, y0), dx-1, dy )
c = dot(grad(x0, y1), dx, dy-1)
d = dot(grad(x1, y1), dx-1, dy-1)
u = fade(dx)
v = fade(dy)
ab = lerp(u, a, b)
cd = lerp(u, c, d)
result = lerp(v, ab, cd)
Single-layer Perlin Noise produces overly “smooth and monotonous” terrain lacking detail. Natural landscapes (mountains, clouds) are multi-scale layered phenomena.
Layer multiple noise octaves with different frequencies and amplitudes:
\[\text{fBm}(x) = \sum_{i=0}^{n} \text{amplitude}_i \cdot \text{noise}(\text{frequency}_i \cdot x)\]| Parameter | Meaning | Typical Value |
|---|---|---|
| Octaves | Number of layers; more = richer detail | 4 ~ 8 |
| Lacunarity | Frequency multiplier per layer | 2.0 |
| Persistence / Gain | Amplitude reduction per layer | 0.5 |
Layer 0: frequency=1, amplitude=1.0 → Large terrain outline
Layer 1: frequency=2, amplitude=0.5 → Mid-scale hills
Layer 2: frequency=4, amplitude=0.25 → Small-scale rocks
Layer 3: frequency=8, amplitude=0.125 → Fine texture
def fbm(x, y, octaves=6, lacunarity=2.0, gain=0.5):
value = 0.0
amplitude = 1.0
frequency = 1.0
for _ in range(octaves):
value += amplitude * perlin(x * frequency, y * frequency)
amplitude *= gain
frequency *= lacunarity
return value
Ken Perlin proposed Simplex Noise in 2001 as an improved version.
| Property | Perlin Noise | Simplex Noise |
|---|---|---|
| Interpolation corners | 2ⁿ corners | n+1 vertices |
| Complexity (high-dim) | O(2ⁿ) | O(n²) |
| Directional artifacts | Yes (axis-aligned) | Less |
| Implementation | Simpler | Complex |
| Patent issues | None | 3D+ patented (use OpenSimplex) |
For high-dimensional scenarios (3D, 4D), Simplex Noise offers dramatic performance gains.
For patent-free implementation, use OpenSimplex2 or SuperSimplex.
import math
import random
def fade(t):
"""Quintic smoothing curve"""
return t * t * t * (t * (t * 6 - 15) + 10)
def lerp(t, a, b):
"""Linear interpolation"""
return a + t * (b - a)
def grad(hash_val, x, y):
"""Select gradient direction and compute dot product"""
h = hash_val & 3
if h == 0: return x + y
if h == 1: return -x + y
if h == 2: return x - y
return -x - y
class PerlinNoise2D:
def __init__(self, seed=None):
random.seed(seed)
# Build permutation table
self.perm = list(range(256))
random.shuffle(self.perm)
self.perm += self.perm # Double for easier indexing
def noise(self, x, y):
# Grid coordinates
xi = int(math.floor(x)) & 255
yi = int(math.floor(y)) & 255
# Local coordinates
xf = x - math.floor(x)
yf = y - math.floor(y)
# Smooth
u = fade(xf)
v = fade(yf)
# Hash values for 4 corners
p = self.perm
aa = p[p[xi ] + yi ]
ab = p[p[xi ] + yi + 1]
ba = p[p[xi + 1] + yi ]
bb = p[p[xi + 1] + yi + 1]
# Interpolate
x1 = lerp(u, grad(aa, xf, yf ), grad(ba, xf-1, yf ))
x2 = lerp(u, grad(ab, xf, yf-1), grad(bb, xf-1, yf-1))
return lerp(v, x1, x2)
# Usage
pn = PerlinNoise2D(seed=42)
for y in range(5):
row = [pn.noise(x * 0.1, y * 0.1) for x in range(10)]
print([f"{v:.2f}" for v in row])
// Hash function (approximation)
float hash(vec2 p) {
p = fract(p * vec2(127.1, 311.7));
p += dot(p, p + 19.19);
return fract(p.x * p.y);
}
// Gradient vectors
vec2 gradient(vec2 p) {
float h = hash(p);
float angle = h * 6.2832; // 0 ~ 2π
return vec2(cos(angle), sin(angle));
}
// 2D Perlin Noise
float perlin(vec2 p) {
vec2 i = floor(p);
vec2 f = fract(p);
vec2 u = f * f * f * (f * (f * 6.0 - 15.0) + 10.0); // fade
float a = dot(gradient(i + vec2(0,0)), f - vec2(0,0));
float b = dot(gradient(i + vec2(1,0)), f - vec2(1,0));
float c = dot(gradient(i + vec2(0,1)), f - vec2(0,1));
float d = dot(gradient(i + vec2(1,1)), f - vec2(1,1));
return mix(mix(a, b, u.x), mix(c, d, u.x), u.y);
}
Height map = fBm(x, z)
Texture blend weight = noise(x, z) → grass/dirt/rock
Cloud density = smoothstep(0.4, 0.6, fBm(x, y, t))
Water height(x, z, t) = noise(x + t, z) + noise(x, z + t * 0.7) * 0.5
Surface normal = gradient of height field
sin(x + noise(x, y) * 10)sin(sqrt(x² + y²) + noise(x, y) * 5)Low frequency (large scale) → Slow variation, large-scale features (mountains, continents)
High frequency (small scale) → Fast variation, fine details (stone texture, grass)
Controls noise value intensity, typically normalized:
# Normalize to [0, 1]
normalized = (noise_value + 1) / 2
2 ~ 3 layers: Smooth, simple (clouds, water)
4 ~ 6 layers: Natural terrain (mountains, hills)
7 ~ 8 layers: Rich detail (rocks, soil)
Note: More layers have diminishing visual returns but linear computation cost.
Advanced technique—warp coordinates using noise for highly natural flow effects:
def warped_noise(x, y):
# Use noise to offset input coordinates
dx = fbm(x + 1.7, y + 9.2)
dy = fbm(x + 8.3, y + 2.8)
return fbm(x + 4.0 * dx, y + 4.0 * dy)
This is the core technique behind many Shadertoy works by Inigo Quilez.
Perlin Noise is not random. Same input always returns same output. It’s deterministic pseudo-randomness. True randomness comes from shuffling the permutation table at initialization (seed).
Theoretical range is $[-\sqrt{n/4}, \sqrt{n/4}]$ (n = dimensions), but actual implementations vary. Don’t assume output strictly constrained to any range—normalize or clamp in practice.
High-frequency noise causes aliasing, producing flickering in dynamic scenes. Choose max frequency based on sampling rate (consult Nyquist theorem).
Many beginners regenerate random gradients on each call, breaking coherence. Pre-build permutation tables, query via lookup.
“自然界中没有真正的随机，只有我们尚未理解的规律。” —— Perlin Noise 的哲学
Perlin Noise（柏林噪声）是由 Ken Perlin 在 1983 年为电影《创：战纪》（Tron）开发的一种梯度噪声算法，并于 1985 年在 SIGGRAPH 论文 An Image Synthesizer 中正式发表。Ken Perlin 也因此获得了奥斯卡科学技术奖。
它是一种相干噪声（Coherent Noise），具有以下特点：
-1 到 1，或归一化到 0 到 1）纯随机（White Noise）：9 4 1 7 3 8 2 5 6 ...
Perlin Noise： 3 4 5 6 5 4 3 2 3 ...
纯随机噪声（白噪声）每个像素完全独立，产生的是”电视雪花”效果，在自然界中几乎不存在。
真实的自然现象（山脉、云朵、火焰、水面）都具有局部相似性：靠近的地方高度/颜色差异不大，但整体上又有宏观的起伏变化。Perlin Noise 正是为了模拟这种特性而生。
| 特性 | 白噪声 | Perlin Noise |
|---|---|---|
| 相邻平滑 | ❌ | ✅ |
| 自然感 | ❌ | ✅ |
| 可控性 | 低 | 高 |
| 计算复杂度 | O(1) | O(2ⁿ)，n 为维度 |
Perlin Noise 的核心思路分三步：
① 建立网格，为每个格点分配随机梯度向量
↓
② 计算采样点到周围格点的距离向量，与梯度向量做点积
↓
③ 用平滑插值函数，将周围格点的点积值融合成最终输出
在每个整数坐标（格点）上，预先分配一个随机的单位方向向量，这就是”梯度”。
与 Value Noise（直接在格点上存随机值）不同，Perlin Noise 存的是方向，而不是值。这使得它的输出更加平滑、视觉上更自然。
普通线性插值（lerp）会产生折线感。Perlin 使用了五次平滑曲线（Quintic Curve）：
这个函数的特点：
早期版本使用的是三次曲线 $3t^2 - 2t^3$（Smoothstep），改进版使用五次曲线（Smootherstep）。
以一维为例，假设采样点 $x = 2.3$：
左格点：x0 = floor(2.3) = 2
右格点：x1 = x0 + 1 = 3
局部坐标：t = 2.3 - 2 = 0.3
一维中，梯度只有两个方向：+1 或 -1，通过置换表（Permutation Table）伪随机确定：
grad(2) = +1
grad(3) = -1
距离向量 × 梯度向量（一维中即乘法）：
左端点贡献：dot(grad(x0), x - x0) = (+1) × (2.3 - 2) = +0.3
右端点贡献：dot(grad(x1), x - x1) = (-1) × (2.3 - 3) = +0.7
u = fade(t) = fade(0.3) ≈ 0.163 （五次曲线）
result = lerp(u, 0.3, 0.7) = 0.3 + 0.163 × (0.7 - 0.3) ≈ 0.365
二维时，采样点 $(x, y)$ 被四个格点围绕：
(x0,y1) -------- (x1,y1)
| . |
| (x,y) |
| |
(x0,y0) -------- (x1,y0)
二维梯度通常从以下 4 个（或 8 个）向量中随机选取：
(1,0), (-1,0), (0,1), (0,-1)
(1,1), (-1,1), (1,-1), (-1,-1) ← 归一化后使用
① 计算采样点到 4 个角的距离向量
② 分别与各角的梯度做点积，得到 4 个值
③ 用 fade(tx) 和 fade(ty) 先沿 x 插值，再沿 y 插值
# 伪代码
a = dot(grad(x0, y0), dx, dy )
b = dot(grad(x1, y0), dx-1, dy )
c = dot(grad(x0, y1), dx, dy-1)
d = dot(grad(x1, y1), dx-1, dy-1)
u = fade(dx)
v = fade(dy)
ab = lerp(u, a, b)
cd = lerp(u, c, d)
result = lerp(v, ab, cd)
单层 Perlin Noise 产生的地形过于”平滑单调”，缺乏细节。自然界的地貌（山脉、云朵）是多尺度叠加的结果。
通过叠加多个频率和振幅不同的 Noise 层（倍频程，Octaves）：
\[\text{fBm}(x) = \sum_{i=0}^{n} \text{amplitude}_i \cdot \text{noise}(\text{frequency}_i \cdot x)\]| 参数 | 说明 | 典型值 |
|---|---|---|
| Octaves（倍频程数） | 叠加的层数，越多细节越丰富 | 4 ~ 8 |
| Lacunarity（缺陷度） | 每层频率的倍增系数 | 2.0 |
| Persistence / Gain | 每层振幅的衰减系数 | 0.5 |
Layer 0: frequency=1, amplitude=1.0 → 大尺度地形轮廓
Layer 1: frequency=2, amplitude=0.5 → 中尺度丘陵
Layer 2: frequency=4, amplitude=0.25 → 小尺度岩石
Layer 3: frequency=8, amplitude=0.125 → 细节纹理
def fbm(x, y, octaves=6, lacunarity=2.0, gain=0.5):
value = 0.0
amplitude = 1.0
frequency = 1.0
for _ in range(octaves):
value += amplitude * perlin(x * frequency, y * frequency)
amplitude *= gain
frequency *= lacunarity
return value
Ken Perlin 于 2001 年提出了 Simplex Noise，作为 Perlin Noise 的改进版本。
| 特性 | Perlin Noise | Simplex Noise |
|---|---|---|
| 插值方向数 | 2ⁿ 个角 | n+1 个顶点 |
| 计算复杂度（高维） | O(2ⁿ) | O(n²) |
| 方向性伪影 | 有（轴对齐） | 较少 |
| 实现复杂度 | 较简单 | 较复杂 |
| 专利问题 | 无 | 3D+ 版本有专利（OpenSimplex 可替代） |
在高维（3D、4D）场景中，Simplex Noise 的性能优势非常显著。
如果需要无专利约束的实现，可以使用 OpenSimplex2 或 SuperSimplex。
import math
import random
def fade(t):
"""五次平滑曲线"""
return t * t * t * (t * (t * 6 - 15) + 10)
def lerp(t, a, b):
"""线性插值"""
return a + t * (b - a)
def grad(hash_val, x, y):
"""根据 hash 值选择梯度方向并计算点积"""
h = hash_val & 3
if h == 0: return x + y
if h == 1: return -x + y
if h == 2: return x - y
return -x - y
class PerlinNoise2D:
def __init__(self, seed=None):
random.seed(seed)
# 构建置换表
self.perm = list(range(256))
random.shuffle(self.perm)
self.perm += self.perm # 复制一份，方便索引
def noise(self, x, y):
# 格点坐标
xi = int(math.floor(x)) & 255
yi = int(math.floor(y)) & 255
# 局部坐标
xf = x - math.floor(x)
yf = y - math.floor(y)
# 平滑
u = fade(xf)
v = fade(yf)
# 四个角的 hash
p = self.perm
aa = p[p[xi ] + yi ]
ab = p[p[xi ] + yi + 1]
ba = p[p[xi + 1] + yi ]
bb = p[p[xi + 1] + yi + 1]
# 插值
x1 = lerp(u, grad(aa, xf, yf ), grad(ba, xf-1, yf ))
x2 = lerp(u, grad(ab, xf, yf-1), grad(bb, xf-1, yf-1))
return lerp(v, x1, x2)
# 使用示例
pn = PerlinNoise2D(seed=42)
for y in range(5):
row = [pn.noise(x * 0.1, y * 0.1) for x in range(10)]
print([f"{v:.2f}" for v in row])
// 置换函数（近似）
float hash(vec2 p) {
p = fract(p * vec2(127.1, 311.7));
p += dot(p, p + 19.19);
return fract(p.x * p.y);
}
// 梯度向量
vec2 gradient(vec2 p) {
float h = hash(p);
float angle = h * 6.2832; // 0 ~ 2π
return vec2(cos(angle), sin(angle));
}
// 2D Perlin Noise
float perlin(vec2 p) {
vec2 i = floor(p);
vec2 f = fract(p);
vec2 u = f * f * f * (f * (f * 6.0 - 15.0) + 10.0); // fade
float a = dot(gradient(i + vec2(0,0)), f - vec2(0,0));
float b = dot(gradient(i + vec2(1,0)), f - vec2(1,0));
float c = dot(gradient(i + vec2(0,1)), f - vec2(0,1));
float d = dot(gradient(i + vec2(1,1)), f - vec2(1,1));
return mix(mix(a, b, u.x), mix(c, d, u.x), u.y);
}
高度图 = fBm(x, z)
纹理混合权重 = noise(x, z) → 草地/泥土/岩石
云密度 = smoothstep(0.4, 0.6, fBm(x, y, t))
水面高度(x, z, t) = noise(x + t, z) + noise(x, z + t * 0.7) * 0.5
法线方向 = gradient of height field
sin(x + noise(x, y) * 10)sin(sqrt(x² + y²) + noise(x, y) * 5)频率低（Scale 大）→ 变化缓慢，大尺度特征（山脉、大陆）
频率高（Scale 小）→ 变化快速，细节特征（石头纹理、草地）
控制 noise 值的强度范围，通常配合归一化：
# 归一化到 [0, 1]
normalized = (noise_value + 1) / 2
2 ~ 3 层：平滑、简洁（适合云朵、水面）
4 ~ 6 层：自然地形（适合山脉、丘陵）
7 ~ 8 层：极丰富细节（适合岩石、土壤）
注意：层数过多在视觉上收益递减，但计算量线性增加。
一种高级技巧，用 noise 对坐标本身进行扭曲，产生极度自然的流动感：
def warped_noise(x, y):
# 用 noise 偏移输入坐标
dx = fbm(x + 1.7, y + 9.2)
dy = fbm(x + 8.3, y + 2.8)
return fbm(x + 4.0 * dx, y + 4.0 * dy)
这正是 Inigo Quilez 等人创作的许多 Shadertoy 作品的核心技巧。
Perlin Noise 不是随机的。相同的输入坐标永远返回相同的值。它是确定性的伪随机过程。真正的随机来自初始化时打乱的置换表（seed）。
理论范围是 $[-\sqrt{n/4}, \sqrt{n/4}]$（n 为维度），实际实现中因梯度选取不同而略有差异。在实践中，不要假设输出严格限制在某个范围内，应对输出进行归一化或 clamp 处理。
高频 noise 容易产生走样（Aliasing），在动态场景中会出现闪烁。应根据采样率选择合适的最高频率（参考奈奎斯特定理）。
random() 直接生成梯度不少初学者在每次调用时重新生成随机梯度，导致输出不连贯或不可重复。正确做法是预先构建置换表，通过查表确定梯度。
Async is not “faster,” but “smarter waiting.”
| Model | Mechanism | Use Case | Drawback |
|---|---|---|---|
| Multiprocessing | Multiple CPU cores in parallel | CPU-intensive (compute, compression) | High memory, expensive context switches |
| Multithreading | OS-level thread scheduling | IO-intensive (with GIL limits) | GIL, race conditions, deadlocks |
| Async IO | Single-threaded event loop | IO-intensive (network, files) | Not suitable for CPU-bound work |
Synchronous mode (one chef cooking one dish):
[Request A sent] → [wait 200ms] → [process] → [Request B sent] → [wait 200ms] → ...
Total time: 200ms × N
Async mode (one chef managing multiple dishes):
[Request A sent] ↘
[Request B sent] → [waiting...] → [A returns, process] → [B returns, process]
[Request C sent] ↗
Total time: ≈ 200ms (longest single request)
Key insight: Async IO’s essence is doing other work while waiting for IO, not true parallel computation.
Functions defined with async def return a coroutine object when called, not executing immediately:
async def say_hello():
print("Hello")
# Calling doesn't execute! Just creates a coroutine object
coro = say_hello()
print(type(coro)) # <class 'coroutine'>
# Must run via event loop
import asyncio
asyncio.run(say_hello()) # This prints Hello
The “control center” of async programs, responsible for:
┌─────────────────────────────────────┐
│ Event Loop │
│ │
│ Task A ──► await IO ──► suspended │
│ Task B ◄── IO done ◄── wake up │
│ Task C ──► await IO ──► suspended │
│ Task A ◄── IO done ◄── wake up │
└─────────────────────────────────────┘
await can only be used inside async def functions. It:
async def fetch():
# Pauses here; event loop runs other tasks
result = await some_async_operation()
# Resumes here after IO completes
return result
Three types of objects can be awaited:
# 1. Coroutine
async def coro(): ...
await coro()
# 2. Task (created via asyncio.create_task)
task = asyncio.create_task(coro())
await task
# 3. Future (low-level, rarely used directly)
future = asyncio.get_event_loop().create_future()
await future
import asyncio
async def main():
print("Start")
await asyncio.sleep(1) # Simulate IO wait
print("Done")
# Python 3.7+ standard entry point
asyncio.run(main())
Async version of time.sleep(), doesn’t block the event loop during wait:
import asyncio
import time
async def task(name, delay):
print(f"{name} start")
await asyncio.sleep(delay) # ✅ Async wait
print(f"{name} done")
async def main():
start = time.time()
# Sequential execution (total time = 1 + 2 = 3s)
await task("A", 1)
await task("B", 2)
print(f"Elapsed: {time.time() - start:.1f}s")
asyncio.run(main())
# Elapsed: 3.0s
import asyncio
import time
async def task(name, delay):
print(f"{name} start")
await asyncio.sleep(delay)
print(f"{name} done")
return f"{name}'s result"
async def main():
start = time.time()
# Create tasks, scheduled immediately (but not yet running)
task_a = asyncio.create_task(task("A", 1))
task_b = asyncio.create_task(task("B", 2))
task_c = asyncio.create_task(task("C", 1))
# Wait for all tasks
result_a = await task_a
result_b = await task_b
result_c = await task_c
print(f"Elapsed: {time.time() - start:.1f}s")
# Elapsed: 2.0s (concurrent; total = longest single task)
asyncio.run(main())
Most common concurrent pattern—waits for all coroutines and collects results:
async def fetch(url):
await asyncio.sleep(1) # Simulate network request
return f"Data from {url}"
async def main():
urls = ["url1", "url2", "url3", "url4", "url5"]
# Execute all requests concurrently
results = await asyncio.gather(
*[fetch(url) for url in urls]
)
for r in results:
print(r)
asyncio.run(main())
# 5 requests sent concurrently; total time ≈ 1s
Exception handling in gather:
# Default: any exception raises immediately
results = await asyncio.gather(coro1(), coro2(), coro3())
# return_exceptions=True: exceptions returned as results, no interruption
results = await asyncio.gather(
coro1(), coro2(), coro3(),
return_exceptions=True
)
for r in results:
if isinstance(r, Exception):
print(f"Error: {r}")
else:
print(f"Success: {r}")
import asyncio
async def task(n):
await asyncio.sleep(n)
return n
async def main():
tasks = [asyncio.create_task(task(i)) for i in [3, 1, 2]]
# Continue when first completes
done, pending = await asyncio.wait(
tasks,
return_when=asyncio.FIRST_COMPLETED
)
for t in done:
print(f"Done: {t.result()}")
# Cancel remaining
for t in pending:
t.cancel()
asyncio.run(main())
return_when Parameter |
Meaning |
|---|---|
ALL_COMPLETED (default) |
Return when all complete |
FIRST_COMPLETED |
Return when first completes |
FIRST_EXCEPTION |
Return when first fails |
async def main():
try:
async with asyncio.timeout(5.0):
await long_running_task()
except TimeoutError:
print("Timed out!")
# Python 3.10 and below: use wait_for
async def main():
try:
result = await asyncio.wait_for(
long_running_task(),
timeout=5.0
)
except asyncio.TimeoutError:
print("Timed out!")
Implement __aenter__ and __aexit__:
class AsyncDBConnection:
async def __aenter__(self):
print("Connecting to database")
self.conn = await create_connection()
return self.conn
async def __aexit__(self, exc_type, exc, tb):
print("Closing connection")
await self.conn.close()
async def main():
async with AsyncDBConnection() as conn:
await conn.execute("SELECT 1")
Implement __aiter__ and __anext__:
class AsyncRange:
def __init__(self, n):
self.n = n
self.i = 0
def __aiter__(self):
return self
async def __anext__(self):
if self.i >= self.n:
raise StopAsyncIteration
await asyncio.sleep(0.1) # Simulate async operation
val = self.i
self.i += 1
return val
async def main():
async for num in AsyncRange(5):
print(num)
Cleaner syntax:
async def async_range(n):
for i in range(n):
await asyncio.sleep(0.1)
yield i
async def main():
async for num in async_range(5):
print(num)
# Also supports async list comprehension
results = [i async for i in async_range(5)]
When multiple coroutines share resources, synchronization is needed (like locks in multithreading).
import asyncio
lock = asyncio.Lock()
shared_resource = 0
async def worker(name):
global shared_resource
async with lock: # Acquire lock; others wait
val = shared_resource
await asyncio.sleep(0.1) # Simulate work
shared_resource = val + 1
print(f"{name}: {shared_resource}")
async def main():
await asyncio.gather(*[worker(f"W{i}") for i in range(5)])
asyncio.run(main())
async def waiter(event, name):
print(f"{name} waiting...")
await event.wait()
print(f"{name} received signal, continuing")
async def setter(event):
await asyncio.sleep(2)
print("Triggering event!")
event.set()
async def main():
event = asyncio.Event()
await asyncio.gather(
waiter(event, "W1"),
waiter(event, "W2"),
setter(event)
)
Most commonly used! Controls max concurrent running coroutines:
async def fetch(session, url, semaphore):
async with semaphore: # Max 10 concurrent
async with session.get(url) as resp:
return await resp.text()
async def main():
semaphore = asyncio.Semaphore(10) # Max concurrency: 10
urls = [f"https://example.com/{i}" for i in range(100)]
async with aiohttp.ClientSession() as session:
tasks = [fetch(session, url, semaphore) for url in urls]
results = await asyncio.gather(*tasks)
import asyncio
async def producer(queue, n):
for i in range(n):
await asyncio.sleep(0.5) # Simulate production
await queue.put(i)
print(f"Produced: {i}, queue size: {queue.qsize()}")
await queue.put(None) # End signal
async def consumer(queue, name):
while True:
item = await queue.get()
if item is None:
queue.task_done()
break
await asyncio.sleep(1) # Simulate consumption
print(f"[{name}] Consumed: {item}")
queue.task_done()
async def main():
queue = asyncio.Queue(maxsize=5) # Max capacity: 5
await asyncio.gather(
producer(queue, 10),
consumer(queue, "Consumer A"),
consumer(queue, "Consumer B"),
)
asyncio.run(main())
pip install aiohttp
import asyncio
import aiohttp
async def fetch(session, url):
async with session.get(url) as response:
return await response.json()
async def main():
async with aiohttp.ClientSession() as session:
data = await fetch(session, "https://api.github.com/users/octocat")
print(data["name"])
asyncio.run(main())
import asyncio
import aiohttp
import time
async def fetch(session, url):
try:
async with session.get(url, timeout=aiohttp.ClientTimeout(total=10)) as resp:
return {"url": url, "status": resp.status, "data": await resp.text()}
except Exception as e:
return {"url": url, "error": str(e)}
async def main():
urls = [
"https://httpbin.org/delay/1",
"https://httpbin.org/delay/2",
"https://httpbin.org/delay/1",
]
start = time.time()
async with aiohttp.ClientSession() as session:
results = await asyncio.gather(*[fetch(session, url) for url in urls])
print(f"Elapsed: {time.time() - start:.1f}s") # ≈ 2s, not 4s
for r in results:
print(r["url"], r.get("status", r.get("error")))
asyncio.run(main())
async def post_data(session, url, payload):
async with session.post(url, json=payload) as resp:
return await resp.json()
async def main():
async with aiohttp.ClientSession() as session:
result = await post_data(
session,
"https://httpbin.org/post",
{"key": "value", "name": "asyncio"}
)
print(result)
Standard open() is synchronous and blocks the event loop. Use aiofiles:
pip install aiofiles
import asyncio
import aiofiles
async def read_file(path):
async with aiofiles.open(path, "r", encoding="utf-8") as f:
return await f.read()
async def write_file(path, content):
async with aiofiles.open(path, "w", encoding="utf-8") as f:
await f.write(content)
async def main():
# Concurrently read multiple files
contents = await asyncio.gather(
read_file("file1.txt"),
read_file("file2.txt"),
read_file("file3.txt"),
)
for content in contents:
print(content[:100])
asyncio.run(main())
pip install asyncpg
import asyncio
import asyncpg
async def main():
# Create connection
conn = await asyncpg.connect(
host="localhost", database="mydb",
user="user", password="password"
)
# Query
rows = await conn.fetch("SELECT id, name FROM users WHERE active = $1", True)
for row in rows:
print(row["id"], row["name"])
# Insert
await conn.execute(
"INSERT INTO users(name, email) VALUES($1, $2)",
"John Doe", "john@example.com"
)
await conn.close()
asyncio.run(main())
async def main():
pool = await asyncpg.create_pool(
host="localhost", database="mydb",
user="user", password="password",
min_size=5, max_size=20
)
async def query(user_id):
async with pool.acquire() as conn:
return await conn.fetchrow("SELECT * FROM users WHERE id = $1", user_id)
# Concurrent queries
results = await asyncio.gather(*[query(i) for i in range(1, 11)])
await pool.close()
pip install aiosqlite
import aiosqlite
async def main():
async with aiosqlite.connect("test.db") as db:
await db.execute("""
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY,
name TEXT
)
""")
await db.execute("INSERT INTO users(name) VALUES(?)", ("John Doe",))
await db.commit()
async with db.execute("SELECT * FROM users") as cursor:
async for row in cursor:
print(row)
Sometimes you need to call async functions from sync code (regular functions, Django views, etc.):
def sync_function():
result = asyncio.run(async_function())
return result
Note:
asyncio.run()creates a new event loop and cannot be called inside an existing event loop.
def sync_function():
loop = asyncio.new_event_loop()
try:
result = loop.run_until_complete(async_function())
finally:
loop.close()
return result
For CPU-intensive or legacy blocking code, use run_in_executor with thread pool:
import asyncio
from concurrent.futures import ThreadPoolExecutor
def blocking_io():
import time
time.sleep(2) # Sync blocking operation
return "done"
def cpu_heavy():
return sum(i * i for i in range(10**7))
async def main():
loop = asyncio.get_event_loop()
# Run in thread pool (for IO-blocking)
result = await loop.run_in_executor(None, blocking_io)
# Run in process pool (for CPU-heavy)
with ProcessPoolExecutor() as pool:
result = await loop.run_in_executor(pool, cpu_heavy)
async def main():
loop = asyncio.get_event_loop() # Get current loop
loop = asyncio.get_running_loop() # Inside coroutine (recommended)
async def main():
loop = asyncio.get_running_loop()
# Execute next iteration (regular function, not coroutine)
loop.call_soon(print, "Execute immediately")
# Delayed execution
loop.call_later(2.0, print, "Execute after 2 seconds")
# Execute at timestamp
loop.call_at(loop.time() + 5.0, print, "Execute after 5 seconds")
await asyncio.sleep(3)
C-based event loop implementation; 2-4x faster than default:
pip install uvloop
import uvloop
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
asyncio.run(main())
import time
import requests # Sync library
# ❌ Wrong: Blocks entire event loop
async def bad_fetch(url):
time.sleep(1) # Blocks!
return requests.get(url) # Blocks!
# ✅ Correct: Use async library
async def good_fetch(url):
await asyncio.sleep(1)
async with aiohttp.ClientSession() as s:
async with s.get(url) as r:
return await r.text()
# ❌ Wrong: coro() just creates coroutine object, never runs
async def main():
result = some_async_func() # Missing await!
print(result) # Prints coroutine object
# ✅ Correct
async def main():
result = await some_async_func()
print(result)
Python will emit: RuntimeWarning: coroutine 'xxx' was never awaited
# ❌ Wrong: Sequential, no concurrency
async def main():
r1 = await fetch("url1") # Wait this
r2 = await fetch("url2") # Then this
r3 = await fetch("url3") # Total time = sum of all
# Total time: 3×delay
# ✅ Correct: True concurrency
async def main():
r1, r2, r3 = await asyncio.gather(
fetch("url1"),
fetch("url2"),
fetch("url3"),
)
# Total time: max(delays)
# ❌ Fails in Jupyter Notebook or existing event loop
asyncio.run(main())
# ✅ Jupyter: directly await
await main()
# Or use nest_asyncio
import nest_asyncio
nest_asyncio.apply()
asyncio.run(main())
# 1. Reuse session; don't create per request
async with aiohttp.ClientSession() as session:
results = await asyncio.gather(*[fetch(session, url) for url in urls])
# 2. Use Semaphore to limit concurrency, prevent overwhelming server
sem = asyncio.Semaphore(20)
async def safe_fetch(url):
async with sem:
return await fetch(url)
# 3. Always handle exceptions; prevent silent task failures
results = await asyncio.gather(*tasks, return_exceptions=True)
# 4. Set timeouts
await asyncio.wait_for(coro(), timeout=30)
# 5. Cancel unwanted tasks
task = asyncio.create_task(coro())
task.cancel()
try:
await task
except asyncio.CancelledError:
pass # Normal cancellation
import asyncio
import aiohttp
import requests
import time
URLS = [f"https://httpbin.org/delay/1" for _ in range(10)]
# ① Sync version
def sync_main():
start = time.time()
results = [requests.get(url).status_code for url in URLS]
print(f"Sync elapsed: {time.time() - start:.1f}s")
# ② Async version
async def async_main():
start = time.time()
async with aiohttp.ClientSession() as session:
async def fetch(url):
async with session.get(url) as r:
return r.status
results = await asyncio.gather(*[fetch(url) for url in URLS])
print(f"Async elapsed: {time.time() - start:.1f}s")
# ③ Rate-limited async
async def async_limited():
sem = asyncio.Semaphore(5)
start = time.time()
async with aiohttp.ClientSession() as session:
async def fetch(url):
async with sem:
async with session.get(url) as r:
return r.status
results = await asyncio.gather(*[fetch(url) for url in URLS])
print(f"Rate-limited async elapsed: {time.time() - start:.1f}s")
sync_main() # Sync elapsed: 10.x s
asyncio.run(async_main()) # Async elapsed: 1.x s
asyncio.run(async_limited()) # Rate-limited: 2.x s
| Method | Time (10 requests, 1s delay each) |
|---|---|
| Sync | ≈ 10s |
| Async (unlimited) | ≈ 1s |
| Async (rate-limited 5) | ≈ 2s |
# Run coroutine
asyncio.run(coro())
# Concurrent wait, collect results
await asyncio.gather(coro1(), coro2(), coro3())
# Create task (schedule immediately)
task = asyncio.create_task(coro())
# Timeout
await asyncio.wait_for(coro(), timeout=5)
# Wait for first completion
done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
# Rate limiting
sem = asyncio.Semaphore(10)
async with sem: ...
# Sync to async (thread pool)
await loop.run_in_executor(None, sync_func, arg1, arg2)
# Get event loop
loop = asyncio.get_running_loop()
# Async sleep
await asyncio.sleep(1)
异步不是”更快”，而是”更聪明地等待”。
| 模型 | 原理 | 适合场景 | 缺点 |
|---|---|---|---|
| 多进程 | 多个 CPU 核心并行 | CPU 密集型（计算、压缩） | 内存占用大，进程切换开销高 |
| 多线程 | 操作系统调度线程 | IO 密集型（但受 GIL 限制） | GIL、竞态条件、死锁 |
| 异步 IO | 单线程事件循环 | IO 密集型（网络、文件） | 不适合 CPU 密集型 |
同步模式（一个厨师只做一道菜）：
[请求A 发出] → [等待 200ms] → [处理结果] → [请求B 发出] → [等待 200ms] → ...
总耗时：200ms × N
异步模式（一个厨师同时照看多道菜）：
[请求A 发出] ↘
[请求B 发出] → [等待中...] → [A 返回，处理] → [B 返回，处理]
[请求C 发出] ↗
总耗时：≈ 200ms（最长单个请求时间）
结论： 异步 IO 的本质是在 等待 IO 的时候去做其他事，而不是真正的并行计算。
用 async def 定义的函数，调用后返回一个协程对象，不会立即执行：
async def say_hello():
print("Hello")
# 调用不会执行！只是创建了协程对象
coro = say_hello()
print(type(coro)) # <class 'coroutine'>
# 必须通过事件循环运行
import asyncio
asyncio.run(say_hello()) # 这才会打印 Hello
异步程序的”调度中心”，负责：
┌─────────────────────────────────────┐
│ Event Loop │
│ │
│ Task A ──► await IO ──► 挂起 │
│ Task B ◄── IO 完成 ◄── 唤醒 │
│ Task C ──► await IO ──► 挂起 │
│ Task A ◄── IO 完成 ◄── 唤醒 │
└─────────────────────────────────────┘
await 只能在 async def 函数内使用，作用是：
async def fetch():
# 暂停在这里，事件循环去运行其他任务
result = await some_async_operation()
# IO 完成后从这里继续
return result
可以被 await 的对象有三类：
# 1. 协程（Coroutine）
async def coro(): ...
await coro()
# 2. Task（asyncio.create_task 创建）
task = asyncio.create_task(coro())
await task
# 3. Future（底层接口，通常不直接使用）
future = asyncio.get_event_loop().create_future()
await future
import asyncio
async def main():
print("开始")
await asyncio.sleep(1) # 模拟 IO 等待
print("结束")
# Python 3.7+ 标准入口
asyncio.run(main())
异步版 time.sleep()，等待期间不阻塞事件循环：
import asyncio
import time
async def task(name, delay):
print(f"{name} 开始")
await asyncio.sleep(delay) # ✅ 异步等待
print(f"{name} 完成")
async def main():
start = time.time()
# 顺序执行（总耗时 = 1 + 2 = 3s）
await task("A", 1)
await task("B", 2)
print(f"耗时：{time.time() - start:.1f}s")
asyncio.run(main())
# 耗时：3.0s
import asyncio
import time
async def task(name, delay):
print(f"{name} 开始")
await asyncio.sleep(delay)
print(f"{name} 完成")
return f"{name} 的结果"
async def main():
start = time.time()
# 创建 Task，立即调度（但还未运行）
task_a = asyncio.create_task(task("A", 1))
task_b = asyncio.create_task(task("B", 2))
task_c = asyncio.create_task(task("C", 1))
# 等待所有 Task 完成
result_a = await task_a
result_b = await task_b
result_c = await task_c
print(f"耗时：{time.time() - start:.1f}s")
# 耗时：2.0s（并发执行，总耗时 = 最长的单个任务）
asyncio.run(main())
最常用的并发方式，等待所有协程完成并收集结果：
async def fetch(url):
await asyncio.sleep(1) # 模拟网络请求
return f"来自 {url} 的数据"
async def main():
urls = ["url1", "url2", "url3", "url4", "url5"]
# 并发执行所有请求
results = await asyncio.gather(
*[fetch(url) for url in urls]
)
for r in results:
print(r)
asyncio.run(main())
# 5 个请求同时发出，总耗时约 1s
gather 的异常处理：
# 默认：任何一个出错会立刻抛出异常
results = await asyncio.gather(coro1(), coro2(), coro3())
# return_exceptions=True：异常作为结果返回，不中断其他任务
results = await asyncio.gather(
coro1(), coro2(), coro3(),
return_exceptions=True
)
for r in results:
if isinstance(r, Exception):
print(f"出错：{r}")
else:
print(f"成功：{r}")
import asyncio
async def task(n):
await asyncio.sleep(n)
return n
async def main():
tasks = [asyncio.create_task(task(i)) for i in [3, 1, 2]]
# 等待第一个完成就继续
done, pending = await asyncio.wait(
tasks,
return_when=asyncio.FIRST_COMPLETED
)
for t in done:
print(f"完成：{t.result()}")
# 取消剩余任务
for t in pending:
t.cancel()
asyncio.run(main())
return_when 参数 |
含义 |
|---|---|
ALL_COMPLETED（默认） |
全部完成才返回 |
FIRST_COMPLETED |
第一个完成就返回 |
FIRST_EXCEPTION |
第一个出错就返回 |
async def main():
try:
async with asyncio.timeout(5.0):
await long_running_task()
except TimeoutError:
print("超时了！")
# Python 3.10 及以下用 wait_for
async def main():
try:
result = await asyncio.wait_for(
long_running_task(),
timeout=5.0
)
except asyncio.TimeoutError:
print("超时了！")
实现 __aenter__ 和 __aexit__：
class AsyncDBConnection:
async def __aenter__(self):
print("连接数据库")
self.conn = await create_connection()
return self.conn
async def __aexit__(self, exc_type, exc, tb):
print("关闭连接")
await self.conn.close()
async def main():
async with AsyncDBConnection() as conn:
await conn.execute("SELECT 1")
实现 __aiter__ 和 __anext__：
class AsyncRange:
def __init__(self, n):
self.n = n
self.i = 0
def __aiter__(self):
return self
async def __anext__(self):
if self.i >= self.n:
raise StopAsyncIteration
await asyncio.sleep(0.1) # 模拟异步操作
val = self.i
self.i += 1
return val
async def main():
async for num in AsyncRange(5):
print(num)
更简洁的写法：
async def async_range(n):
for i in range(n):
await asyncio.sleep(0.1)
yield i
async def main():
async for num in async_range(5):
print(num)
# 也可以用列表推导
results = [i async for i in async_range(5)]
多个协程共享资源时需要同步控制（类比多线程的锁）。
import asyncio
lock = asyncio.Lock()
shared_resource = 0
async def worker(name):
global shared_resource
async with lock: # 获取锁，其他协程等待
val = shared_resource
await asyncio.sleep(0.1) # 模拟操作
shared_resource = val + 1
print(f"{name}: {shared_resource}")
async def main():
await asyncio.gather(*[worker(f"W{i}") for i in range(5)])
asyncio.run(main())
async def waiter(event, name):
print(f"{name} 等待事件...")
await event.wait()
print(f"{name} 收到事件，继续执行")
async def setter(event):
await asyncio.sleep(2)
print("触发事件！")
event.set()
async def main():
event = asyncio.Event()
await asyncio.gather(
waiter(event, "W1"),
waiter(event, "W2"),
setter(event)
)
最常用！限制同时运行的协程数量：
async def fetch(session, url, semaphore):
async with semaphore: # 最多 10 个同时执行
async with session.get(url) as resp:
return await resp.text()
async def main():
semaphore = asyncio.Semaphore(10) # 最大并发 10
urls = [f"https://example.com/{i}" for i in range(100)]
async with aiohttp.ClientSession() as session:
tasks = [fetch(session, url, semaphore) for url in urls]
results = await asyncio.gather(*tasks)
import asyncio
async def producer(queue, n):
for i in range(n):
await asyncio.sleep(0.5) # 模拟生产耗时
await queue.put(i)
print(f"生产：{i}，队列大小：{queue.qsize()}")
await queue.put(None) # 结束信号
async def consumer(queue, name):
while True:
item = await queue.get()
if item is None:
queue.task_done()
break
await asyncio.sleep(1) # 模拟消费耗时
print(f"[{name}] 消费：{item}")
queue.task_done()
async def main():
queue = asyncio.Queue(maxsize=5) # 最大容量 5
await asyncio.gather(
producer(queue, 10),
consumer(queue, "消费者A"),
consumer(queue, "消费者B"),
)
asyncio.run(main())
pip install aiohttp
import asyncio
import aiohttp
async def fetch(session, url):
async with session.get(url) as response:
return await response.json()
async def main():
async with aiohttp.ClientSession() as session:
data = await fetch(session, "https://api.github.com/users/octocat")
print(data["name"])
asyncio.run(main())
import asyncio
import aiohttp
import time
async def fetch(session, url):
try:
async with session.get(url, timeout=aiohttp.ClientTimeout(total=10)) as resp:
return {"url": url, "status": resp.status, "data": await resp.text()}
except Exception as e:
return {"url": url, "error": str(e)}
async def main():
urls = [
"https://httpbin.org/delay/1",
"https://httpbin.org/delay/2",
"https://httpbin.org/delay/1",
]
start = time.time()
async with aiohttp.ClientSession() as session:
results = await asyncio.gather(*[fetch(session, url) for url in urls])
print(f"耗时：{time.time() - start:.1f}s") # ≈ 2s，而非 4s
for r in results:
print(r["url"], r.get("status", r.get("error")))
asyncio.run(main())
async def post_data(session, url, payload):
async with session.post(url, json=payload) as resp:
return await resp.json()
async def main():
async with aiohttp.ClientSession() as session:
result = await post_data(
session,
"https://httpbin.org/post",
{"key": "value", "name": "asyncio"}
)
print(result)
标准的 open() 是同步的，会阻塞事件循环。使用 aiofiles：
pip install aiofiles
import asyncio
import aiofiles
async def read_file(path):
async with aiofiles.open(path, "r", encoding="utf-8") as f:
return await f.read()
async def write_file(path, content):
async with aiofiles.open(path, "w", encoding="utf-8") as f:
await f.write(content)
async def main():
# 并发读取多个文件
contents = await asyncio.gather(
read_file("file1.txt"),
read_file("file2.txt"),
read_file("file3.txt"),
)
for content in contents:
print(content[:100])
asyncio.run(main())
pip install asyncpg
import asyncio
import asyncpg
async def main():
# 创建连接
conn = await asyncpg.connect(
host="localhost", database="mydb",
user="user", password="password"
)
# 查询
rows = await conn.fetch("SELECT id, name FROM users WHERE active = $1", True)
for row in rows:
print(row["id"], row["name"])
# 插入
await conn.execute(
"INSERT INTO users(name, email) VALUES($1, $2)",
"张三", "zhangsan@example.com"
)
await conn.close()
asyncio.run(main())
async def main():
pool = await asyncpg.create_pool(
host="localhost", database="mydb",
user="user", password="password",
min_size=5, max_size=20
)
async def query(user_id):
async with pool.acquire() as conn:
return await conn.fetchrow("SELECT * FROM users WHERE id = $1", user_id)
# 并发查询
results = await asyncio.gather(*[query(i) for i in range(1, 11)])
await pool.close()
pip install aiosqlite
import aiosqlite
async def main():
async with aiosqlite.connect("test.db") as db:
await db.execute("""
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY,
name TEXT
)
""")
await db.execute("INSERT INTO users(name) VALUES(?)", ("张三",))
await db.commit()
async with db.execute("SELECT * FROM users") as cursor:
async for row in cursor:
print(row)
有时需要在同步代码（如普通函数、Django 视图）中调用异步函数：
def sync_function():
result = asyncio.run(async_function())
return result
注意：
asyncio.run()会创建新的事件循环，不能在已有事件循环中调用。
def sync_function():
loop = asyncio.new_event_loop()
try:
result = loop.run_until_complete(async_function())
finally:
loop.close()
return result
CPU 密集或遗留的同步阻塞代码，用 run_in_executor 放到线程池：
import asyncio
from concurrent.futures import ThreadPoolExecutor
def blocking_io():
import time
time.sleep(2) # 同步阻塞操作
return "done"
def cpu_heavy():
return sum(i * i for i in range(10**7))
async def main():
loop = asyncio.get_event_loop()
# 放到线程池（适合 IO 阻塞）
result = await loop.run_in_executor(None, blocking_io)
# 放到进程池（适合 CPU 密集）
with ProcessPoolExecutor() as pool:
result = await loop.run_in_executor(pool, cpu_heavy)
async def main():
loop = asyncio.get_event_loop() # 获取当前循环
loop = asyncio.get_running_loop() # 只在协程中使用（推荐）
async def main():
loop = asyncio.get_running_loop()
# 下一次迭代时执行（非协程，是普通函数）
loop.call_soon(print, "马上执行")
# 延迟执行
loop.call_later(2.0, print, "2秒后执行")
# 指定时间戳执行
loop.call_at(loop.time() + 5.0, print, "5秒后执行")
await asyncio.sleep(3)
用 C 实现的事件循环，性能比默认提升 2~4 倍：
pip install uvloop
import uvloop
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())
asyncio.run(main())
import time
import requests # 同步库
# ❌ 错误：阻塞整个事件循环
async def bad_fetch(url):
time.sleep(1) # 阻塞！
return requests.get(url) # 阻塞！
# ✅ 正确：使用异步库
async def good_fetch(url):
await asyncio.sleep(1)
async with aiohttp.ClientSession() as s:
async with s.get(url) as r:
return await r.text()
# ❌ 错误：coro() 只是创建了协程对象，根本没有运行
async def main():
result = some_async_func() # 漏掉了 await！
print(result) # 打印的是协程对象
# ✅ 正确
async def main():
result = await some_async_func()
print(result)
Python 会发出 RuntimeWarning: coroutine 'xxx' was never awaited 警告。
# ❌ 错误：顺序执行，没有并发
async def main():
r1 = await fetch("url1") # 等这个完成
r2 = await fetch("url2") # 再执行这个
r3 = await fetch("url3") # 总耗时 = 三者之和
# ✅ 正确：真正并发
async def main():
r1, r2, r3 = await asyncio.gather(
fetch("url1"),
fetch("url2"),
fetch("url3"),
)
# ❌ 在 Jupyter Notebook 或已有事件循环的环境中会报错
asyncio.run(main())
# ✅ Jupyter 中直接 await
await main()
# 或使用 nest_asyncio
import nest_asyncio
nest_asyncio.apply()
asyncio.run(main())
# 1. Session 复用，不要每次请求都创建
async with aiohttp.ClientSession() as session:
results = await asyncio.gather(*[fetch(session, url) for url in urls])
# 2. 用 Semaphore 控制并发量，避免打垮服务器
sem = asyncio.Semaphore(20)
async def safe_fetch(url):
async with sem:
return await fetch(url)
# 3. 总是处理异常，避免 Task 静默失败
results = await asyncio.gather(*tasks, return_exceptions=True)
# 4. 设置超时
await asyncio.wait_for(coro(), timeout=30)
# 5. 取消不需要的 Task
task = asyncio.create_task(coro())
task.cancel()
try:
await task
except asyncio.CancelledError:
pass # 正常取消
import asyncio
import aiohttp
import requests
import time
URLS = [f"https://httpbin.org/delay/1" for _ in range(10)]
# ① 同步版本
def sync_main():
start = time.time()
results = [requests.get(url).status_code for url in URLS]
print(f"同步耗时：{time.time() - start:.1f}s")
# ② 异步版本
async def async_main():
start = time.time()
async with aiohttp.ClientSession() as session:
async def fetch(url):
async with session.get(url) as r:
return r.status
results = await asyncio.gather(*[fetch(url) for url in URLS])
print(f"异步耗时：{time.time() - start:.1f}s")
# ③ 限制并发的异步版本
async def async_limited():
sem = asyncio.Semaphore(5)
start = time.time()
async with aiohttp.ClientSession() as session:
async def fetch(url):
async with sem:
async with session.get(url) as r:
return r.status
results = await asyncio.gather(*[fetch(url) for url in URLS])
print(f"限流异步耗时：{time.time() - start:.1f}s")
sync_main() # 同步耗时：10.x s
asyncio.run(async_main()) # 异步耗时：1.x s
asyncio.run(async_limited()) # 限流异步：2.x s
| 方式 | 耗时（10个各延迟1s的请求） |
|---|---|
| 同步 | ≈ 10s |
| 异步（无限制） | ≈ 1s |
| 异步（并发限 5） | ≈ 2s |
# 运行协程
asyncio.run(coro())
# 并发等待，收集结果
await asyncio.gather(coro1(), coro2(), coro3())
# 创建 Task（立即调度）
task = asyncio.create_task(coro())
# 超时
await asyncio.wait_for(coro(), timeout=5)
# 等待第一个完成
done, pending = await asyncio.wait(tasks, return_when=asyncio.FIRST_COMPLETED)
# 限流
sem = asyncio.Semaphore(10)
async with sem: ...
# 同步转异步（线程池）
await loop.run_in_executor(None, sync_func, arg1, arg2)
# 获取事件循环
loop = asyncio.get_running_loop()
# 异步 sleep
await asyncio.sleep(1)