Version	Description	Date
1.0.1	Initial document	25 Dec 2024

I. Server requirement

1. Web server

xStreamer supports almost OS: Windows, MacOS, Linux. However we recommend to use Linux, document is applied for linux system only.

For production environment, xStreamer needs a VPS server with at least:

4GB of RAM - recommend 8GB
40GB of HDD
2 CPU core
3 domains / sub domains
- api.[your-domain] and point to server IP address
- admin.[your-domain] and point to server IP address
- [your-domain] and point to server IP address

II. Software requirements

xStreamer architecture needs these softwares

NodeJS LTS version >= v20.x
- To install please download NodeJS here
MongoDB >= v4.2
- To install please download mongoDB
Redis server >= v3.2
- To install please download and setup redis
FFMPEG
Nginx >= v1.3
PM2 is a daemon process manager that will help you manage and keep your application online 24/7
Yarn to manage nodeJS package

Ensure all softwares above are running before setup source code

Helpful links

Testing

From command line / terminal please run these commands to check

1. NodeJS

$ node -v
v16.8.2

2. MongoDB

$ mongo --version
MongoDB shell version v4.2.7
git version: 1b82c812a9c0bbf6dc79d5400de9ea99e6ffa025
allocator: system
modules: none
build environment:
    distarch: x86_64
    target_arch: x86_64

3. Redis server

$ redis-server --version
Redis server v=5.0.4 sha=00000000:0 malloc=libc bits=64 build=d4ba11298acbb366

4. FFMPEG

$ ffmpeg -version
fmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
libavutil      54. 31.100 / 54. 31.100
libavcodec     56. 60.100 / 56. 60.100
libavformat    56. 40.101 / 56. 40.101
libavdevice    56.  4.100 / 56.  4.100
libavfilter     5. 40.101 /  5. 40.101
libavresample   2.  1.  0 /  2.  1.  0
libswscale      3.  1.101 /  3.  1.101
libswresample   1.  2.101 /  1.  2.101
libpostproc    53.  3.100 / 53.  3.100

5. NGINX

$ nginx -v
nginx version: nginx/1.10.3

6. Pm2

$ pm2 -v
[PM2] Spawning PM2 daemon with pm2_home=/Users/xxx/.pm2
[PM2] PM2 Successfully daemonized
4.4.0

7. Yarn

$ yarn -v
1.16.0

III. Setup and test project locally

We provide source code with 4 folders, check below and use in the app accordingly

front-office web frontend
back-office admin control panel for the application
api api of the application
config-sample
- env product environment example config
  - api.env sample configuration for api
  - front-office.env sample configuration for front web
  - back-office.env sample configuration for admin web
- nginx
  - admin.example.com.conf sample nginx configuration for admin web
  - api.example.com.conf sample nginx configuration for api
  - example.com.conf sample nginx configuration for web frontend

1. API

Ensure node v16.x, MongoDB, Redis server, FFMPEG and yarn are running
CD to your API folder
Run yarn insall --production=false to install nodeJS dependencies and dev dependencies

$ yarn
yarn install v1.16.0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...

Create new .env from config-sample > env > api.env file in the root dir

Content .env looks like below

# development or production
NODE_ENV=production

# http port to use, default is 8080
PORT=8080

# secret key to hash data
JWT_SECRET=57637278384742980983526417468101

# mongodb conection string. Read more https://www.mongodb.com/docs/manual/reference/connection-string/
MONGODB_URI=mongodb://localhost/xstreamers

# Redis configuration for queue
# configuration can be change in src > config > configuration.ts
REDIS_QUEUE_HOST=127.0.0.1
REDIS_QUEUE_PORT=6379
## username and password of redis if any. Support from Redis 6
REDIS_QUEUE_USERNAME=
REDIS_QUEUE_PASSWORD=
REDIS_QUEUE_DB=0
## enter true/1 if use TLS. For custom redis service such as AWS redis cluster
REDIS_QUEUE_TLS=
## enter true/1 if use cluster mode. For custom redis service such as AWS redis cluster
REDIS_QUEUE_USE_CLUSTER_MODE=

# base url for file upload server
# it can be the same api server or separate file server if any
FILE_UPLOAD_BASE_URL=https://api.example.com/files/{fileId}/upload
# base url for file service, it can be api service if using same server
FILE_BASE_URL=https://api.example.com

# your domain name, use to seed default data
DOMAIN=example.com

# API base url
BASE_URL=https://api.example.com

# email template dir
TEMPLATE_DIR=templates

# default redirect URL once clicking on verification on email
EMAIL_VERIFIED_SUCCESS_URL=https://example.com/auth/email-verified-success

Check Migration point to seed default
Run yarn dev to start development env

$ yarn dev
:48 PM] Starting compilation in watch mode...

[2:49:13 PM] Found 0 errors. Watching for file changes.

[Nest] 60068   - 08/03/2020, 2:49:17 PM   [NestFactory] Starting Nest application...
[Nest] 60068   - 08/03/2020, 2:49:17 PM   [InstanceLoader] QueueModule dependencies initialized +134ms
[Nest] 60068   - 08/03/2020, 2:49:17 PM   [InstanceLoader] RedisModule dependencies initialized +0ms
[Nest] 60068   - 08/03/2020, 2:49:17 PM   [InstanceLoader] HttpModule dependencies initialized +1ms
[Nest] 60068   - 08/03/2020, 2:49:17 PM   [InstanceLoader] QueueModule dependencies initialized +0ms
[Nest] 60068   - 08/03/2020, 2:49:17 PM   [InstanceLoader] ServeStaticModule dependencies initialized +4ms
....

Open browsers and run http://localhost:8080 it should show success message or Hello World!

2. Frontend web (front office)

Ensure nodejs and yarn are running
From frontend root directory run yarn insall --production=false to install nodeJS dependencies and dev dependencies

$ yarn
yarn install v1.16.0
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...

Create .env file from config-sample > env > front-office.env. Content looks like below. Change it with your configuration accordingly

# configuration for web. If have any change in configuration we have to rebuild application and reload.
## development or production
NODE_ENV=production

## api endpoint in server side
API_ENDPOINT=http://localhost:8080

## api endpoint in client side
NEXT_PUBLIC_API_ENDPOINT=https://api.example.com

# custom secret key for authen
NEXTAUTH_SECRET=G47OMY1US7BECpcr0X-H8llMeKp4ldt7tmiEAYi7wrOoJNiJ8RKfz5We1nai4wd8n53AS19Axvp7jQZBc1zoIA
# your frontoffice url
NEXTAUTH_URL=https://example.com

Run yarn run dev to start development env

$ yarn run dev
yarn run v1.16.0
$ ts-node --project tsconfig.server.json server/index.ts
Warning: Built-in CSS support is being disabled due to custom CSS configuration being detected.
See here for more info: https://err.sh/next.js/built-in-css-disabled

> Using external babel configuration

Then you can open browser http://localhost:8081 to see the web

3. Admin web

Similar Frontend web, same steps.
Create .env file from config-sample > env > back-office.env. Content looks like below. Change it with your configuration accordingly

# configuration for web. If have any change in configuration we have to rebuild application and reload.
## development or production
NODE_ENV=production

## api endpoint in server side
API_ENDPOINT=http://localhost:8080

## api endpoint in client side
NEXT_PUBLIC_API_ENDPOINT=https://api.example.com

# front office endpoint, use to clear cache or get direct url if any
FRONT_OFFICE_ENDPOINT=http://localhost:8081
# back office endpoint, use to clear cache or get direct url if any
BACK_OFFICE_ENDPOINT=http://localhost:8082

Run yarn dev to start development env

$ yarn run dev
yarn run v1.16.0
$ ts-node --project tsconfig.server.json server/index.ts
Warning: Built-in CSS support is being disabled due to custom CSS configuration being detected.
See here for more info: https://err.sh/next.js/built-in-css-disabled

> Using external babel configuration

Then you can open browser http://localhost:8082 to see the web

IV. Setup and setup production environment with nginx

1. API

Make sure you have installed nodeJS dependencies by yarn install --production=false command. Configure it like local setup, pm2 was installed
In API root directory, run yarn build

$ yarn build
yarn run v1.16.0
$ rimraf dist
$ nest build
✨  Done in 22.62s

Testing by run command yarn start:prod or node dist/main.js

$ yarn start
yarn run v1.16.0
$ node dist/main
[Nest] 60205   - 08/03/2020, 2:56:11 PM   [NestFactory] Starting Nest application...
....

Run pm2 start yarn --interpreter bash --name api.example.com -- start to run application under port 8080

Note: If have error when starting with Pm2, please try to start without interpreter

pm2 start yarn --name api.example.com -- start

2. Frontend web

cd user web folder
Make sure you have installed nodeJS dependencies by yarn install --production=false command. Configure it like local setup, pm2 was installed
Run yarn build

$ yarn build
yarn run v1.16.0
$ next build && tsc && tsc --project tsconfig.server.json
Warning: Built-in CSS support is being disabled due to custom CSS configuration being detected.
See here for more info: https://err.sh/next.js/built-in-css-disabled

> Using external babel configuration
> Location: "/Users/tuong/Projects/frontend-base-reactjs/.babelrc"
Creating an optimized production build  

Compiled with warnings.

chunk styles [mini-css-extract-plugin]
Conflicting order between:
 * css ./node_modules/css-loader??ref--6-1!./node_modules/less-loader/dist/cjs.js??ref--6-2!./src/components/video/video.less
 * css ./node_modules/css-loader??ref--6-1!./node_modules/less-loader/dist/cjs.js??ref--6-2!./src/components/common/layout/page.less
 * css ./node_modules/css-loader??ref--6-1!./node_modules/less-loader/dist/cjs.js??ref--6-2!./src/components/common/base/loader.less
 ....

Testing by run yarn start it should show success message

$ yarn start
yarn run v1.16.0
next start
ready - started server on http://localhost:8081

Run pm2 start yarn --interpreter bash --name api.example.com -- start to run application under port 8081. You can change port in package.json in start script

Note: If have error when starting with Pm2, please try to start without interpreter

pm2 start yarn --name example.com -- start

3. Admin web

cd user admin folder
Make sure you have installed nodeJS dependencies by yarn install --production=false command. Configure it like local setup, pm2 was installed
Run yarn build

Testing by run yarn start it should show success message
Run pm2 start yarn --interpreter bash --name admin.example.com -- start to run application under port 8082. You can change port in package.json in start script

Note: If have error when starting with Pm2, please try to start without interpreter

pm2 start yarn --name admin.example.com -- start

4. Setup nginx

a. Api

Let say our frontend code is in /var/www/api.example.com folder and api is running under port 8080, add new file in /etc/nginx/sites-enabled/api.example.com and content as bellow
You have to change with our server name and source code path

server {
  listen 80;
	root /var/www/api.example.com/public;

	index index.html index.htm;
  server_name api.example.com;
  gzip on;
  gzip_proxied any;
  gzip_comp_level 4;
  gzip_types text/css application/javascript image/svg+xml;

  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;
  sendfile_max_chunk 512;
  client_max_body_size 2000M;

  access_log off;
  error_log off;

	location / {
    proxy_set_header   X-Forwarded-For $remote_addr;
    proxy_set_header   Host $http_host;
    proxy_pass         http://localhost:8080;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_cache_bypass $http_upgrade;
	}

  location /.well-known {
    alias /var/www/api.example.com/public/.well-known;
  }

  # check if url is a protected folder such as privated videos
  # then check auth
	location ~/protected/ {
	  auth_request /authcheck;
	  root /var/www/api.example.com/public/;
	}

	location = /authcheck {
	  internal;
    set $query '';
    if ($request_uri ~* "[^\?]+\?(.*)$") {
      set $query $1;
    }
    proxy_pass http://localhost:8080/files/auth/check?$query;
    proxy_pass_request_body   off;
    proxy_set_header Content-Length "";
	}
}

b. Frontend

Let say our frontend code is in /var/www/example.com folder and Frontend app is running under port 8081, add new file in /etc/nginx/sites-enabled/example.com and content as bellow
You have to change with our server name and source code path

server {
  listen 80;
	server_name example.com;

	root /var/www/example.com/public;

  gzip on;
  gzip_proxied any;
  gzip_comp_level 4;
  gzip_types text/css application/javascript image/svg+xml;

  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;
  sendfile_max_chunk 512;

  # tuning performance for log request
  access_log off;
  error_log off;

	location / {
    proxy_pass http://localhost:8081/;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_cache_bypass $http_upgrade;
    proxy_redirect        off;
    proxy_set_header      Host $host;
    proxy_set_header      X-Real-IP $remote_addr;
    proxy_set_header      X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header      Proxy "";
  }

  location /.well-known {
    alias /var/www/example.com/public/.well-known;
  }

  location /_next/static/ {
    alias /var/www/example.com/dist/.next/static/$1;
    access_log off;
    expires max;
  }
}

c. Admin

Let say our frontend code is in /var/www/admin.example.com folder and Frontend app is running under port 8082, add new file in /etc/nginx/sites-enabled/admin.example.com and content as bellow
You have to change with our server name and source code path

server {
  listen 80;
	server_name admin.example.com;
	root /var/www/admin.example.com/public;

  gzip on;
  gzip_proxied any;
  gzip_comp_level 4;
  gzip_types text/css application/javascript image/svg+xml;

  sendfile on;
  tcp_nopush on;
  tcp_nodelay on;
  keepalive_timeout 65;
  types_hash_max_size 2048;
  sendfile_max_chunk 512;

  access_log off;
  error_log off;

	location / {
    proxy_set_header   X-Forwarded-For $remote_addr;
    proxy_set_header   Host $http_host;
    proxy_pass         http://localhost:8082;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_cache_bypass $http_upgrade;
  }

  location /.well-known {
    alias /var/www/admin.example.com/public/.well-known;
  }

  location /_next/static/ {
    alias /var/www/admin.example.com/dist/.next/static/$1;
    access_log off;
    expires max;
  }
}

d. Testing

Open browser and access: http://yourdomain.com/ to access web frontend
Open browser and access: http://admin.yourdomain.com to access web admin

5. Check applications are running under pm2

Run pm2 ls to see your applications
Run pm2 stop [id] to stop app or pm2 reload [id] to reload
Run applications in start up by pm2 startup then pm2 save

6. Set up nginx with https

Check here to setup https with nginx
Setup https with Certbot nginx

7. Migration

From API root directory run yarn migrate to seed default values for settings and users
- Default admin: admin@[yourdomain] with domain is in .env file
- Default password: adminadmin
- If admin password is wrong, you can run node script reset-admin-pw to reset it to adminadmin
You can change default values in src > migrations folder

Installation documentation

I. Server requirement

1. Web server

II. Software requirements

Helpful links

Testing

1. NodeJS

2. MongoDB

3. Redis server

4. FFMPEG

5. NGINX

6. Pm2

7. Yarn

III. Setup and test project locally

1. API

2. Frontend web (front office)

3. Admin web

IV. Setup and setup production environment with nginx

1. API

2. Frontend web

3. Admin web

4. Setup nginx

a. Api

b. Frontend

c. Admin

d. Testing

5. Check applications are running under pm2

6. Set up nginx with https

7. Migration