1. Home
  2. Docs
  3. Engineering
  4. NestJS

NestJS

We use NestJS to create business logic services and workers (excluding repositories, which are implemented using Strapi).

Logging

Use winston. We only use limited feature set like setting log levels, because we follow 12-factor app principle, the log routing should be handled by the deployment environment.

For format, using JSON structured logging is probably a good idea. See: https://medium.com/unomaly/logging-wisdom-how-to-log-5a19145e35ec.

The logs eventually need to go into Elasticsearch.

For container environment like AWS Fargate or Kubernetes, you can use Filebeat, or maybe pipe AWS CloudWatch Logs to Elasticsearch somehow.

For AWS Lambda, I think you need to configure AWS CloudWatch Logs to pipe to Elasticsearch.

Fastify instead of Express

Warning: Fastify + GraphQL is currently unusable due to nestjs/graphql#1205.

Reference: Performance (Fastify) – NestJS

Add @nestjs/platform-fastify package:

yarn remove @nestjs/platform-express apollo-server-express @types/express
yarn add @nestjs/platform-fastify
# if you use GraphQL
yarn add apollo-server-fastify

Edit src/main.ts:

import { config } from 'dotenv';
config();

import { NestFactory } from '@nestjs/core';
import { FastifyAdapter, NestFastifyApplication } from '@nestjs/platform-fastify';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(
    AppModule, new FastifyAdapter(), {
    logger: ['debug', 'log', 'error', 'warn', 'verbose'],
  });
  // const appModule = app.get(AppModule);
  // await appModule.init();
  await app.listen(parseInt(process.env.PORT || '3001'));
}
bootstrap();

In Fastify, it’s called FastifyReply instead of Express’s Response:

import { FastifyReply } from 'fastify';

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Post('generatePdf')
  async generatePdf(@Body() data: CertificateData, @Res() res: FastifyReply) {
    if (!data?.awardeeName) {
      throw new HttpException('data is required', 400);
    }
    const doc = await generateCertificate(data);
    res.header('Content-Type', 'application/pdf');
    // doc.pipe(res);
    const buffers = [];
    doc.on('data', data => buffers.push(data));
    doc.on('end', () => {
      const buf = Buffer.concat(buffers);
      console.info(`Rendered PDF size is ${buf.length} bytes`);
      res.send(buf);
    });
    doc.end();
  }

}

JWT Authentication with GraphQL & Subscriptions Support

Reference: nestjs/graphql#48

Additional dependency we need is @nestjs/jwt:

yarn add @nestjs/jwt

Create Auth module:

yarn nest g module auth

Define auth/fusionauth-payload.ts:

/**
 * Holds information about current user.
 */
export class FusionAuthPayload {
  aud: string;
  exp?: number;
  lat?: number;
  iss: string;
  sub: string;
  jti: string;
  authenticationType: string;
  email: string;
  email_verified: boolean;
  preferred_username: string;
  applicationId: string;
  roles: string[];
}

Create JwtAuth service:

yarn nest g service JwtAuth auth
import { Injectable, UnauthorizedException } from '@nestjs/common';
import { JwtService } from '@nestjs/jwt';
import { FusionAuthPayload } from '../fusionauth-payload';

@Injectable()
export class JwtAuthService {
  constructor(private jwtService: JwtService) {}

  // async validateUser(email: string, password: string): Promise {
  //   const user = await this.usersRepository.findOne({ email });
  //   if (!user) {
  //     throw 'user not found';
  //   }
  //   if (user.password === password) {
  //     const token = this.jwtService.sign({
  //       userId: user.id,
  //       email: user.email,
  //     });
  //     return { user, token };
  //   }
  // }

  async validateToken(token: string): Promise {
    try {
      console.debug('Verifying JWT token:', token, '...');
      const payload: FusionAuthPayload = this.jwtService.verify(token);
      console.debug('JWT payload:', payload);
      return payload;
    } catch (e) {
      console.warn('Invalid JWT token:', e);
      throw new UnauthorizedException(`Invalid JWT token: ${e}`);
    }
  }
}

Create GqlAuth guard:

yarn nest g guard GqlAuth auth
import { CanActivate, ExecutionContext, Injectable, BadRequestException, UnauthorizedException } from '@nestjs/common';
import { Observable } from 'rxjs';
import { GqlExecutionContext } from '@nestjs/graphql';
import { JwtAuthService } from './jwt-auth/jwt-auth.service';
import { Reflector } from '@nestjs/core';
import { FusionAuthPayload } from './fusionauth-payload';

export const ROLE_SALES_MANAGER = 'Sales Manager';

/**
 * Allow only specified roles:
 * @SetMetadata('roles', ['Sales Manager'])
 */
@Injectable()
export class GqlAuthGuard implements CanActivate {
  constructor(private reflector: Reflector,
    private readonly jwtAuthService: JwtAuthService) {}

  getRequest(context: ExecutionContext) {
    const ctx = GqlExecutionContext.create(context);
    return ctx.getContext().req;
  }

  async canActivate(context: ExecutionContext): Promise {
    const req = this.getRequest(context);
    const authHeader = req.headers.authorization as string;

    if (!authHeader) {
      throw new BadRequestException('Authorization header not found.');
    }
    const [type, token] = authHeader.split(' ');
    if (type !== 'Bearer') {
      throw new BadRequestException(`Authentication type \'Bearer\' required. Found \'${type}\'`);
    }
    let user: FusionAuthPayload;
    try {
      user = req.user = await this.jwtAuthService.validateToken(token);
    } catch (e) {
      throw new UnauthorizedException(`Cannot validate JWT token: ${e}`);
    }

    // get from 'roles' metadata in method (want to fallback to class, but how?)
    const allowedRoles = this.reflector.get('roles', context.getHandler());
    if (allowedRoles) {
      const intersection = allowedRoles.filter(_ => (user.roles || []).includes(_));
      if (intersection.length >= 1) {
        return true;
      } else {
        throw new UnauthorizedException(`${context.getClass().name}.${context.getHandler().name} requires ${allowedRoles}, but user '${user.preferred_username}' only has these roles: ${user.roles}`);
      }
    } else {
      const msg = `No role is allowed by ${context.getClass().name}.${context.getHandler().name}`;
      console.warn(msg);
      throw new UnauthorizedException(msg);
    }
  }
}

Configure JwtModule in auth/auth.module.ts:

import { Module } from '@nestjs/common';
import { JwtModule } from '@nestjs/jwt';
import { JwtAuthService } from './jwt-auth/jwt-auth.service';
import pemtools from 'pemtools';

let publicKey: string | Buffer | null = process.env.JWT_PUBLIC_KEY;
if (process.env.JWT_PUBLIC_KEY && !process.env.JWT_PUBLIC_KEY.startsWith('-----')) {
  publicKey = Buffer.from(process.env.JWT_PUBLIC_KEY, 'base64');
  // See: https://github.com/nestjs/jwt/issues/447
  publicKey = pemtools(publicKey, 'PUBLIC KEY').toString();
}
// console.debug('publicKey=', publicKey);

@Module({
  imports: [
    JwtModule.register({
      publicKey,
      secret: process.env.JWT_SECRET,
    })
  ],
  exports: [JwtAuthService],
  providers: [JwtAuthService],
})
export class AuthModule {}

You’ll need to provide JWT_PUBLIC_KEY (recommended for FusionAuth access token for server-side verification, as it can be verified without knowing a shared secret).

To use the guard, you must put 'roles' metadata on the method (not the class):

@UseGuards(GqlAuthGuard)
@Resolver(of => User)
export class UsersResolver {
...
  @Query(returns => [User])
  @SetMetadata('roles', [ROLE_SALES_MANAGER])
  async users(

JWT Authentication using @nestjs/passport (not recommended)

See nestjs/graphql#48 for discussion why using @nestjs/passport is more trouble than it’s worth.

Reference: NestJS > Authentication > JWT functionality

Example implementation is in lovia-billing.

Add passport, passport-jwt, @nestjs/passport, @nestjs/jwt packages:

yarn add passport passport-jwt @nestjs/passport @nestjs/jwt

Generate module auth:

nest g module auth

Create auth/jwt-user.ts that holds information of current user (optional if you don’t need the current user, because you can just use the JWT payload):

/**
 * Holds information about current user.
 */
export class JwtUser {
  idV4: string;
  username: string;
  ssoId?: string;
  email?: string;
  roles: string[];
  userIsFinanceManager: boolean;
}

Create src/jwt.strategy.ts that will use ExtractJwt:

import { Injectable } from "@nestjs/common";
import { PassportStrategy } from "@nestjs/passport";
import { ExtractJwt, Strategy } from "passport-jwt";
import { JwtUser } from "./jwt-user";

@Injectable()
export class JwtStrategy extends PassportStrategy(Strategy) {
  constructor() {
    super({
      // Bearer header only, ignore URL query parameter:
      // jwtFromRequest: ExtractJwt.fromAuthHeaderAsBearerToken(),
      jwtFromRequest: ExtractJwt.fromExtractors([
        ExtractJwt.fromAuthHeaderAsBearerToken(),
        ExtractJwt.fromUrlQueryParameter('accessToken')
      ]),
      ignoreExpiration: false,
      // either HMAC shared secret, or PEM-encoded RSA public key
      secretOrKey: process.env.JWT_SECRET || process.env.JWT_PUBLIC_KEY,
    });
  }

  /**
   * Convert JWT payload into our own structure... or you can just return the payload.
   * @param payload 
   */
  async validate(payload: any): Promise {
    const user = {
      idV4: payload.sub,
      username: payload.username,
      //ssoId: 
      email: payload.email,
      roles: payload.roles || [],
      // helper
      userIsFinanceManager: (payload.roles || []).includes('Finance Manager'),
    };
    console.debug('JWT user:', user);
    return user;
  }

}

Now configure PassportModule, JwtModule with JWT_SECRET_OR_KEY environment variable, and use our JwtStrategy in auth/auth.module.ts:

import { Module } from '@nestjs/common';
import { PassportModule } from '@nestjs/passport';
import { JwtModule } from '@nestjs/jwt';
import { JwtStrategy } from './jwt.strategy';

@Module({
  imports: [
    PassportModule,
    JwtModule.register({
      secret: process.env.JWT_SECRET,
      publicKey: process.env.JWT_PUBLIC_KEY,
      signOptions: {},
    })
  ],
  providers: [JwtStrategy],
})
export class AuthModule {}

Create the JwtAuth guard inside auth module:

yarn nest g guard JwtAuth auth

Implement auth/jwt-auth.guard.ts by just extending AuthGuard:

import { Injectable } from '@nestjs/common';
import { AuthGuard } from '@nestjs/passport';

@Injectable()
export class JwtAuthGuard extends AuthGuard('jwt') {
}

Now the guard is ready to use.

To use the guard from your controller or controller methods, add @UseGuards(JwtAuthGuard) and access the payload (or returned value of JwtStrategy.validate()) using req.user:

import { FastifyRequest } from 'fastify';

...

  @UseGuards(JwtAuthGuard)
  @Get('profile')
  getProfile(@Request() req: FastifyRequest) {
    const user = (req as any).user as JwtUser;
    return user;
  }

Generate JWT Token

yarn global add njwt
# to enable node to require yarn global modules, add the following to ~/.bash_profile
export NODE_PATH=$(yarn global dir)/node_modules
node
const jwt = require('njwt');
let claims = { sub: 'finance', username: 'finance', roles: ["Finance Manager"] };
let token = jwt.create(claims, 'top-secret-phrase');
token.setExpiration(new Date().getTime() + 60*1000*60*24*365*100);
token.compact();

Now you can use generated token in Postman by adding Bearer token, as part of HTTP request header:

Authorization: Bearer YOUR_TOKEN_HERE

Reference: https://developer.okta.com/blog/2018/11/13/create-and-verify-jwts-with-node

Deploy NestJS-Express with Serverless Framework on AWS Lambda

Prepare & Test Serverless Framework Locally

Reference:

yarn global add serverless

Add required packages:

yarn add aws-lambda aws-serverless-express
yarn add --dev @types/aws-lambda @types/aws-serverless-express

Create src/lambda.ts:

import { Handler, Context } from 'aws-lambda';
import { Server } from 'http';
import { createServer, proxy } from 'aws-serverless-express';
import { eventContext } from 'aws-serverless-express/middleware';

import { NestFactory } from '@nestjs/core';
import { ExpressAdapter } from '@nestjs/platform-express';
import { AppModule } from './app.module';

const express = require('express');

// NOTE: If you get ERR_CONTENT_DECODING_FAILED in your browser, this
// is likely due to a compressed response (e.g. gzip) which has not
// been handled correctly by aws-serverless-express and/or API
// Gateway. Add the necessary MIME types to binaryMimeTypes below
const binaryMimeTypes: string[] = ['application/octet-stream', 'application/pdf'];

let cachedServer: Server;

// Create the Nest.js server and convert it into an Express.js server
async function bootstrapServer(): Promise {
  if (!cachedServer) {
     const expressApp = express();
     const nestApp = await NestFactory.create(AppModule, new
ExpressAdapter(expressApp))
     nestApp.use(eventContext());
     await nestApp.init();
     cachedServer = createServer(expressApp, undefined,
binaryMimeTypes);
  }
  return cachedServer;
}

// Export the handler : the entry point of the Lambda function
export const handler: Handler = async (event: any, context: Context) => {
  cachedServer = await bootstrapServer();
  return proxy(cachedServer, event, context, 'PROMISE').promise;
}

Serverless plugins: serverless-plugin-typescript.

yarn add --dev serverless-plugin-typescript serverless-offline

Create serverless.yml:

service:
  name: certificate2pdf

package:
  exclude:
    # Exclude "built" files since we're using serverless-plugin-typescript
    - dist/**
  include:
    - templates/**

plugins:
  - serverless-plugin-typescript
  # to transpile and minify your code
  # - serverless-plugin-opttimize
  # to be able to test your app offline
  - serverless-offline
  # - serverless-plugin-warmup

provider:
  name: aws
  runtime: nodejs12.x
  stage: ${opt:stage, 'dev'} # Set the default stage used. Default is dev
  region: ${opt:region, 'ap-southeast-1'} # Overwrite the default region used. Default is us-east-1
  # RAM usage is only about 150 MiB, but need bigger RAM to speed up CPU to avoid timeout
  memorySize: 2048 # Overwrite the default memory size. Default is 1024
  timeout: 15 # The default is 6 seconds. Note: API Gateway current maximum is 30 seconds
  apiGateway:
    # minimumCompressionSize: 1024 # Compress response when larger than specified size in bytes (must be between 0 and 10485760)
    binaryMediaTypes: # Optional binary media types the API might return
      - application/octet-stream
      - application/pdf

functions:
  main: # The name of the lambda function
    # The module 'handler' is exported in the file 'src/lambda'
    handler: src/lambda.handler
    events:
      - http:
          method: any
          path: /{any+}

# custom:
#   customDomain:
#     domainName: certificate2pdf.talentiva.net
#     # certificateName: '*.talentiva.net'
#     # Since this API is used only internally by ERPNext, save bandwidth & time by using Regional endpoint instead of Edge
#     endpointType: regional
#     # basePath: ''
#     stage: ${self:provider.stage}
#     createRoute53Record: false
#   # Enable warmup on all functions (only for production and staging)
#   warmup:
#     enabled: true

Try locally:

# You'll need to either export required environment variables, or use dotenv package
export JWT_PUBLIC_KEY=...
~/.yarn/bin/serverless offline start

Deploy “dev” stage to AWS Lambda with Generic URL

During serverless deploy: You’ll get error: messageText: “Option ‘–incremental’ can only be specified using tsconfig, emitting to single file or when option --tsBuildInfoFile is specified.”, so edit tsconfig.json and add inside compileOptions:

    "tsBuildInfoFile": ".tsbuildinfo"

To actually deploy, set required environment variables first using AWS Systems Manager Parameter Store.

Important: Unless you specify a region in serverless.yml, it will use us-east-1, even if ~/.aws/config specifies a different default region.

Using in Gitpod: You may want to create ~/.aws/credentials, or alternatively export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY before running deploy.

~/.yarn/bin/serverless deploy

To deploy a different environment/stage, e.g. production or staging:

~/.yarn/bin/serverless deploy -s production

Note: Binary Media Types (PDF, etc.)

  • In Fastify/Express, ensure to set binaryMimeTypes, e.g. return awsLambdaFastify(fastifyApp, { binaryMimeTypes: ['application/octet-stream', 'application/pdf'] }); Fastify doesn’t seem to support wildcards here.
  • In serverless.yml, set binaryMediaTypes to */*
  • In client, you must set Accept: application/pdf or Accept: application/*. Unfortunately, using Accept: */* will not work, and you’ll get Base64.
  • In serverless.yml, “contentHandling: CONVERT_TO_BINARY” doesn’t seem to be needed, and seems to cause a warning that it is ignored.

Deploy NestJS-Fastify with Serverless Framework on AWS Lambda

Before doing deploying Fastify to AWS Lambda, make sure you’ve converted your NestJS project to Fastify (above).

Prepare & Test Serverless Framework Locally

Reference:

yarn global add serverless

Add aws-lambda-fastify package:

yarn add @types/aws-lambda aws-lambda aws-lambda-fastify

Add to src/global.d.ts:

declare module 'aws-lambda-fastify';

Create src/lambda.ts:

import { Context } from 'aws-lambda';
import awsLambdaFastify from 'aws-lambda-fastify';
import { AppModule } from './app.module';
import { FastifyAdapter, NestFastifyApplication } from '@nestjs/platform-fastify';
import { fastify } from 'fastify';
import { NestFactory } from '@nestjs/core';

let cachedProxy: (event: any, context: Context) => Promise;

async function bootstrap(): Promise<(event: any, context: Context) => Promise> {
  const fastifyApp = fastify();

  const app = await NestFactory.create(
    AppModule, new FastifyAdapter(fastifyApp));
  await app.init();

  return awsLambdaFastify(fastifyApp, { binaryMimeTypes: ['application/octet-stream', 'application/pdf'] });
}

export async function handler(event: any, context: Context): Promise {
  if (!cachedProxy) {
    const proxy = await bootstrap();
    cachedProxy = proxy;
  }

  return cachedProxy(event, context);
}

Serverless plugins: serverless-plugin-typescript.

yarn add --dev serverless-plugin-typescript serverless-offline

You’ll get error: messageText: “Option ‘–incremental’ can only be specified using tsconfig, emitting to single file or when option --tsBuildInfoFile is specified.”, so edit tsconfig.json and add:

    "tsBuildInfoFile": ".tsbuildinfo"

Create serverless.yml:

service:
  name: certificate2pdf

package:
  exclude:
    - dist/**
  include:
    - templates/**

plugins:
  - serverless-plugin-typescript
  # to transpile and minify your code
  # - serverless-plugin-opttimize
  # to be able to test your app offline
  - serverless-offline
  # - serverless-domain-manager
  # - serverless-plugin-warmup

provider:
  name: aws
  runtime: nodejs12.x
  stage: ${opt:stage, 'dev'} # Set the default stage used. Default is dev
  region: ${opt:region, 'ap-southeast-1'} # Overwrite the default region used. Default is us-east-1
  # RAM usage is only about 150 MiB, but need bigger RAM to speed up CPU to avoid timeout
  memorySize: 2048 # Overwrite the default memory size. Default is 1024
  timeout: 15 # The default is 6 seconds. Note: API Gateway current maximum is 30 seconds
  apiGateway:
    # minimumCompressionSize: 1024 # Compress response when larger than specified size in bytes (must be between 0 and 10485760)
    binaryMediaTypes: # Optional binary media types the API might return
      - application/octet-stream
      - application/pdf

functions:
  main: # The name of the lambda function
    # The module 'handler' is exported in the file 'src/lambda'
    handler: src/lambda.handler
    events:
      - http:
          method: any
          path: /{any+}

# custom:
#   # Enable warmup on all functions (only for production and staging)
#   warmup:
#     enabled: true

Try locally:

# You'll need to either export required environment variables, or use dotenv package
export JWT_SECRET=...
serverless offline start

Troubleshooting & Logging

serverless logs -f main --tail -s dev
serverless logs -f main --tail -s production

GraphQL schema.gql issue

Error: EROFS: read-only file system, open ‘schema.gql’

Solution: https://stackoverflow.com/a/61048144/122441

TL;DR: in src/app.module.ts:

    GraphQLModule.forRoot({
      installSubscriptionHandlers: false,
      // See: https://stackoverflow.com/a/61048144/122441
      autoSchemaFile: process.env.NODE_ENV === 'development' ? 'schema.gql' : true,
    }),
...

Deploy to AWS Lambda with Custom Domain & SSL Certificate

Reference: https://www.serverless.com/blog/serverless-api-gateway-domain

  1. Request SSL certificate using AWS ACM, make sure to use wildcard domain + naked domain.
yarn add --dev serverless-domain-manager

Add in plugins, and also custom.customDomain configuration in serverless.yml:

plugins:
  # ...
  - serverless-domain-manager

custom:
  customDomain:
    domainName: certificate2pdf.talentiva.net
    # certificateName: '*.talentiva.net'
    # Since this API is used only internally by ERPNext, save bandwidth & time by using Regional endpoint instead of Edge
    endpointType: regional
    # basePath: ''
    stage: ${self:provider.stage}
    createRoute53Record: false
#   # Enable warmup on all functions (only for production and staging)
#   warmup:
#     enabled: true
~/.yarn/bin/serverless create_domain -s production

Then deploy again:

~/.yarn/bin/serverless deploy -s production

Use Cloudflare/DNS Manager to add CNAME record for the custom domain.

Testing API Webhooks/Callbacks

It’s recommended to use Gitpod whenever possible.

Otherwise, when developing locally you can use ngrok to get a public HTTPS endpoint.

ngrok http --region=ap 3005
Was this article helpful to you? Yes No

How can we help?

Leave a Reply

Your email address will not be published. Required fields are marked *