Building Scalable APIs with Node.js and Express

Building APIs that can handle growth is crucial for any successful application. In this guide, we'll explore the best practices for creating scalable, maintainable APIs using Node.js and Express.

Architecture Principles

1. Layered Architecture

Organize your code into distinct layers:

src/
  controllers/    # Handle HTTP requests
  services/       # Business logic
  models/         # Data models
  middleware/     # Custom middleware
  routes/         # Route definitions
  utils/          # Utility functions

2. Separation of Concerns

Each layer should have a single responsibility:

// controllers/userController.js
const userService = require('../services/userService')

exports.getUser = async (req, res) => {
  try {
    const user = await userService.getUserById(req.params.id)
    res.json(user)
  } catch (error) {
    res.status(500).json({ error: error.message })
  }
}

// services/userService.js
const User = require('../models/User')

exports.getUserById = async (id) => {
  const user = await User.findById(id)
  if (!user) {
    throw new Error('User not found')
  }
  return user
}

Performance Optimization

1. Database Optimization

• Use connection pooling
• Implement proper indexing
• Use query optimization

// Database connection with pooling
const mongoose = require('mongoose')

mongoose.connect(process.env.MONGODB_URI, {
  maxPoolSize: 10,
  serverSelectionTimeoutMS: 5000,
  socketTimeoutMS: 45000,
})

2. Caching Strategies

Implement caching at multiple levels:

const redis = require('redis')
const client = redis.createClient()

// Cache middleware
const cache = (duration) => {
  return async (req, res, next) => {
    const key = req.originalUrl
    const cached = await client.get(key)
    
    if (cached) {
      return res.json(JSON.parse(cached))
    }
    
    res.sendResponse = res.json
    res.json = (body) => {
      client.setex(key, duration, JSON.stringify(body))
      res.sendResponse(body)
    }
    
    next()
  }
}

3. Rate Limiting

Protect your API from abuse:

const rateLimit = require('express-rate-limit')

const limiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // limit each IP to 100 requests per windowMs
  message: 'Too many requests from this IP'
})

app.use('/api/', limiter)

Error Handling

Implement comprehensive error handling:

// Global error handler
app.use((error, req, res, next) => {
  const status = error.statusCode || 500
  const message = error.message
  const data = error.data
  
  res.status(status).json({
    message: message,
    data: data
  })
})

// Async error wrapper
const asyncHandler = (fn) => (req, res, next) => {
  Promise.resolve(fn(req, res, next)).catch(next)
}

Security Best Practices

1. Input Validation
2. Authentication & Authorization
3. HTTPS Only
4. CORS Configuration
5. Security Headers

const helmet = require('helmet')
const cors = require('cors')

app.use(helmet())
app.use(cors({
  origin: process.env.ALLOWED_ORIGINS?.split(',') || 'http://localhost:3000',
  credentials: true
}))

Testing Strategy

Implement comprehensive testing:

// Unit test example
describe('User Service', () => {
  test('should get user by id', async () => {
    const mockUser = { id: '1', name: 'John Doe' }
    User.findById = jest.fn().mockResolvedValue(mockUser)
    
    const result = await userService.getUserById('1')
    expect(result).toEqual(mockUser)
  })
})

Deployment Considerations

1. Environment Configuration
2. Process Management (PM2)
3. Load Balancing
4. Monitoring & Logging
5. Health Checks

Conclusion

Building scalable APIs requires careful planning and adherence to best practices. Focus on clean architecture, performance optimization, proper error handling, and comprehensive testing.

Remember that scalability is not just about handling more requests—it's about maintaining code quality and developer productivity as your application grows.