When a company plans an AI knowledge base for about 100 employees, the first infrastructure question is usually simple: what server specs do we need? The tempting answer is a fixed hardware list. In reality, there is no universal configuration that fits every 100-person organization. A knowledge base for HR policy Q&A is very different from a customer support assistant that handles thousands of external questions. A system using external model APIs is very different from one running local embedding and large language models. A pilot with a few thousand documents is very different from a production deployment with heavy workflow integration.
The better way to think about server specs is capacity planning. You need to understand document volume, user concurrency, model strategy, retrieval architecture, workflow complexity, uptime expectations, and operational responsibility. A platform such as FastGPT can help teams build knowledge-based AI applications, but the infrastructure plan still depends on how the organization deploys and uses the system. The goal is not to buy the largest server possible. The goal is to match resources to the actual workload and leave a responsible path for growth.
Start with the Workload, Not the Headcount
One hundred employees does not mean one hundred people will use the knowledge base at the same time. In many internal deployments, daily active users may be a fraction of total employees, and true simultaneous usage may be much lower. But some scenarios create bursts. HR onboarding, policy updates, product training, customer service shifts, and sales proposal deadlines can all increase usage suddenly. The infrastructure plan should estimate concurrency, not only total user count.
Ask how the knowledge base will be used. Is it an internal assistant for employee self-service? Is it used by customer service agents throughout the day? Will it be embedded in a customer-facing portal? Will it run automated workflows? Will users upload many documents, or will administrators manage content centrally? These answers change the server requirements more than the employee count itself.
A useful planning exercise is to divide users into three groups: occasional users, daily users, and operational users. Occasional users may ask a few policy or process questions each week. Daily users may rely on the assistant for sales, HR, IT, or support work. Operational users may keep the system open for much of the day and expect fast answers. A 100-person company with ten operational support agents may need more reliable capacity than a 300-person company where the assistant is used only for occasional HR self-service. Headcount is a weak proxy. Usage pattern is the real driver.
Separate the Core System from Model Inference
The biggest infrastructure variable is whether model inference runs locally. If the system uses external model APIs, the private server mainly supports application services, document processing, databases, vector storage, retrieval, workflow execution, logs, and administration. In that case, CPU, memory, disk I/O, network stability, and database performance are often more important than GPU capacity.
If the company requires local models, the calculation changes. Local embedding models and large language models may require GPU resources, more memory, model serving infrastructure, and careful latency planning. A small internal knowledge base can become much heavier if every generation request runs on local hardware. The team should decide early which components must be local: document storage, embeddings, vector database, reranking, large language model inference, or all of them.
This decision should be documented as part of the architecture. For example, one organization may decide that documents, embeddings, and vector indexes must stay inside the intranet, while generation can use an approved external model API. Another may require every component to run locally because retrieved context cannot leave the network. These two designs may look similar to users, but their server requirements are very different. Treat model placement as a first-class infrastructure decision, not a detail to resolve after procurement.
Estimate Document Volume and Indexing Load
Document volume affects storage, indexing time, retrieval quality, and maintenance work. A 100-person company may have only a few hundred curated documents, or it may have years of manuals, tickets, policies, and project files. The system must parse documents, split them into chunks, generate embeddings, store metadata, and update indexes when content changes. Large PDFs, tables, mixed-language files, and duplicated versions can increase processing cost.
For planning, estimate the number of documents, average file size, update frequency, and expected growth over the next year. Also estimate how much of the content is actually useful. Uploading every file in a shared drive is rarely a good idea. Curated documents improve retrieval quality and reduce infrastructure waste. A smaller, well-maintained knowledge base often performs better than a massive unmanaged archive.
Indexing load is also different from query load. A company may have light daily Q&A traffic but heavy monthly document updates. Another may rarely upload new documents but serve many daily questions. If document ingestion and user chat run on the same resources, large indexing jobs can slow the user experience. For production, plan how ingestion jobs are scheduled, whether they run in the background, and whether they can be paused or retried without affecting active users.
Plan CPU, Memory, and Storage by Component
The application layer needs stable CPU and memory for web services, background jobs, workflow execution, and administration. Document ingestion can create short bursts of CPU and memory usage, especially when parsing large files. Vector search and metadata filtering require memory and storage performance. Logs and conversation history require predictable disk capacity. Backups also need space.
Instead of asking for one server spec, split the system into components: application service, database, vector storage, object/file storage, model services, monitoring, and backup. Small deployments may run several components on one machine during a pilot. Production deployments often separate critical services so that ingestion, search, and user-facing chat do not interfere with each other. Separation also makes upgrades and troubleshooting easier.
Memory planning deserves special attention. Retrieval systems often benefit from enough memory to keep hot indexes, metadata, and database cache responsive. Storage planning should account for original files, parsed text, embeddings, indexes, logs, backups, and historical versions. Disk speed can matter during ingestion and search. A server with enough raw capacity but poor I/O may feel slow under real workloads. For an enterprise knowledge base, stability and predictable latency are usually more valuable than impressive peak benchmark numbers.
Think Carefully About GPU Requirements
GPU requirements depend almost entirely on local model strategy. If all generation and embedding calls use external APIs, a GPU may not be required for the knowledge base server. If embeddings run locally, GPU may improve throughput but may not always be mandatory depending on the model and usage pattern. If large language model inference runs locally, GPU planning becomes central.
Do not choose a GPU plan based only on model size. Consider concurrency, context length, expected response speed, quantization strategy, model serving framework, and peak usage. A model that works for one user in a demo may not handle multiple users smoothly. Also consider operational expertise. Local model serving requires monitoring, updates, memory management, and fallback planning. For a 100-person company, external models or hybrid models may be more practical unless there is a strong data-boundary requirement.
If the team is unsure, test both paths. Run the knowledge base with an external model and measure user value first. Then test local inference for the same question set and compare latency, answer quality, operations effort, and cost. The decision should be evidence-based. Buying GPU capacity before proving the use case can lock the project into a heavier operating model than the business actually needs.
Include Database and Vector Search Performance
An AI knowledge base depends on retrieval. The system must search relevant chunks quickly, filter by metadata, and return evidence for the model. If retrieval is slow, users experience the whole assistant as slow even if the model is fast. If retrieval is poorly configured, answers become vague or wrong. Infrastructure planning should therefore include database and vector search performance, not only model inference.
Important factors include index size, query volume, metadata filters, reranking, and concurrent searches. If the knowledge base serves multiple departments with different permissions, retrieval may need filtering by role, department, document type, or customer. These filters can affect performance. The team should test retrieval with real documents and real questions before finalizing production infrastructure.
Design for Backups, Logs, and Monitoring
A 100-person deployment may sound small, but once employees depend on the assistant, it becomes a production system. Backups are needed for documents, metadata, application configuration, workflows, and database state. Logs are needed for troubleshooting, quality review, and audit. Monitoring is needed for uptime, latency, error rates, storage growth, and model provider failures.
These operational needs consume resources. Logs can grow quickly if every conversation, retrieval result, and workflow execution is stored. Backups require storage and testing. Monitoring requires agents, dashboards, and alert rules. A serious server plan should include these from the beginning rather than adding them after the first failure.
Use a Pilot to Validate Sizing
The most reliable way to size infrastructure is to run a pilot with realistic usage. Choose one knowledge domain, upload real documents, create a test question set, and invite a small group of users. Measure ingestion time, search latency, answer latency, memory usage, CPU load, storage growth, and user concurrency. Then extrapolate cautiously.
The pilot should include peak scenarios. Upload a batch of new documents. Ask several users to test at the same time. Run common workflows. Test bad questions and refusal behavior. Review logs. This gives the team actual evidence instead of relying on generic server recommendations. It also reveals whether the bottleneck is model inference, retrieval, document parsing, or workflow execution.
After the pilot, convert observations into a simple capacity plan. Record average response time, peak response time, failed requests, CPU and memory usage, index growth, storage growth, and administrator workload. Then define thresholds for expansion. For example, add more worker capacity if ingestion jobs delay user queries, separate databases if search latency rises, or add model-serving capacity if generation becomes the bottleneck. This turns server sizing into an ongoing operating practice instead of a one-time guess.
How FastGPT Fits Infrastructure Planning
FastGPT’s official documentation can help teams understand how knowledge applications are organized and what components need attention during deployment. During planning, evaluate the exact mode you intend to use: external models, local models, private deployment, workflow integration, and document ingestion. Each mode changes resource needs.
The most useful evaluation is not “can the system run?” but “can the system run comfortably under our real workload?” Test document updates, concurrent usage, retrieval latency, citations, and administrative operations. A good deployment plan leaves room for growth without overspending on capacity that the team cannot operate.
Final Takeaway
For a 100-person enterprise knowledge base, server specs should be derived from workload, not headcount. The key questions are how many users will be active at the same time, how many documents will be indexed, whether models run locally, how retrieval is configured, how workflows are used, and what level of monitoring and backup is required. A small internal assistant using external models can be relatively light. A private deployment with local models, heavy documents, and workflow integrations needs a more serious architecture.
The best approach is staged. Start with a focused pilot, measure real usage, identify bottlenecks, and then scale the infrastructure. This avoids both underbuilding a system that users cannot trust and overbuilding a system before value is proven. Server planning is not a one-time purchase decision. It is part of the operating model for making enterprise knowledge usable every day.