Observability
Pychat is an opensource absolutely free communication tool targeted for a company use. It's created as alternative to Slack/Discord. See the table below to understand its key features.
Pychat | Slack | Skype | Telegram | Viber | Discord | |
---|---|---|---|---|---|---|
Open Source | + | - | - | - | - | - |
Free | + | +/- | +/- | + | +/- | +/- |
Screen sharing | + | + | - | + | - | + |
Stream drawing | + | - | - | - | - | - |
Syntax highlight | + | - | - | - | - | + |
Only company users | + | + | - | - | - | + |
Audio/Video conference | + | + | + | + | - | + |
Can run on your server | + | - | - | - | - | - |
Audio/Video messages | + | - | - | + | + | - |
P2P file sharing | + | - | - | - | - | - |
P2P messaging | + | - | - | + | - | - |
Message read status | + | - | + | + | + | - |
Tagging user | + | + | - | + | + | + |
Message threads | + | + | - | - | - | - |
PWA (works w/o lan) | + | - | - | - | - | - |
Desktop client | +/- | + | + | + | +/- | + |
Mobile client | +/- | + | + | + | + | + |
3rd-party plugins | - | + | - | - | - | + |
I would personally use discord or slack as a company chat. They are built and maintained by thousands of people rather than a single person. BUT wait!!! There're some key factors of picking pychat over others:
Notice: pychat is migrating from vue2 to vue3 and this change has been released to master. The older code that supports some feature (electron/cordova) is still not migrated and located at branch vue2-webpack
Please don't use this build for production, as it uses debug ssl certificate, lacks a few features and all files are located inside of container, meaning you will lose all data on container destroy.
docker run -tp 443:443 deathangel908/pychat-test
Please run each step very carefully. Do not skip editing files, reading comments or any instructions. This may lead to bugs in the future.
server.key
and certificate.crt
. If you don't own a domain you can create self-signed certificates with command below, with self-signed certificate browser will warn users with broken ssl.openssl req -nodes -new -x509 -keyout server.key -out certificate.crt -days 3650
wget https://raw.githubusercontent.com/akoidan/pychat/master/backend/chat/settings_example.py
wget https://raw.githubusercontent.com/akoidan/pychat/master/docker/pychat.org/production.json
wget https://raw.githubusercontent.com/akoidan/pychat/master/docker/pychat.org/turnserver.conf
settings_example.py
according comments in it.turnserver.conf
docker volume create pychat_data
containerid=`docker container create --name dummy -v pychat_data:/data hello-world`
docker cp settings_example.py dummy:/data/settings.py
docker cp production.json dummy:/data/production.json
docker cp turnserver.conf dummy:/data/turnserver.conf
docker cp certificate.crt dummy:/data/certificate.crt
docker cp server.key dummy:/data/server.key
docker rm dummy
This volume will contain all production data: config, mysql data, redis and etc. If you need to edit files inside container you can use:
docker run -i -t -v pychat_data:/tmp -it alpine /bin/sh
docker run -t -v pychat_data:/data -p 443:443 -p 3478:3478 deathangel908/pychat
If you don't or unable to run docker you can alway do the setup w/o it. You definitely spend more time, so I would recommend to use docker if possible. But if you're still sure, here's the setup for cent-os/archlinux based system:
/srv/http/pychat
. If you want to close the project into a different directory, replace all absolute paths in config files. You can use download_content.sh rename_root_directory
to do that.pacman -S postfix gcc jansson
.alias yum="python2 $(which yum)"
to /etc/bashrc
if you use python3. And then install that packages yum install python34u, python34u-pip, redis, mysql-server, mysql-devel, postfix, mailx
nginx_upload_module
written in C
) instead of python uploader (which is a lot slower) you should build nginx yourself. For archlinux setup requires pacman -S python-lxml gd make geoip
. To build nginx with this module run from the root user: bash download_content.sh build_nginx 1.15.3 2.3.0
. And create dir + user useradd nginx; install -d -m 0500 -o http -g http /var/cache/nginx/
. If you don't, just install nginx with your package manager: e.g. pacman -S nginx
or yum install nginx
on centospychat.org
, you want to replace all occurrences of pychat.org
in rootfs directory for your domain. To simplify replacing use my script: ./download_content.sh rename_domain your.new.domain.com
. Also check rootfs/etc/nginx/sites-enabled/pychat.conf
if server_name
section is correct after renaming.openssl req -nodes -new -x509 -keyout server.key -out certificate.crt -days 3650
/etc/nginx/sites-enabled/pychat.conf
and modify it by:server_name
to one matching your domain/ip addressssl_certificate
and ssl_certificate_key
path to ones that you generatedupload_file
module, remove locations api/upload_file
and @upload_file
, otherwise leave it as it is.sh download_content.sh copy_root_fs
.mkdir backend/downloading_photos
in the backend directory and give it access chmod 777 downloading_photos
cDon't forget to change the owner of current (project) directory to http
user: chown -R http:http
. And reload systemd config systemctl daemon-reload
. Also youinstall -d -m 0555 -o postfix -g postfix /etc/postfix/virtual; postmap /etc/postfix/virtual; newaliases; touch /etc/postfix/virtual-regexp; echo 'root postmaster' > /etc/aliases
packages=( mysqld redis tornado@8888 nginx postfix ) ; for package in "${packages[@]}" ; do systemctl enable $package; done;
. Service mysqld
could be named mysql
on Ubuntu.packages=( redis-server nginx postfix mysqld tornado@8888) ; for package in "${packages[@]}" ; do service $package start; done;
packages=( redis nginx postfix mysqld tornado) ; for package in "${packages[@]}" ; do systemctl start $package; done;
chkconfig mysqld on; chkconfig on; chkconfig tornado on; chkconfig redis on; chkconfig postfix on
pychat/backend/logs
directory.sudo journalctl -u YOUR_SERVICE
. Where YOUR_SERVICE could be: nginx
, mysql
, tornado
http
has access to you project directory, and all directories inside, especially to /photos
cd frontend; nvm install; nvm use
.yarn install --frozen-lockfile
cp docker/pychat.org/production.json ./frontend/
yarn build
. This generates static files in frotnend/dist
directory.Pychat uses websql and built the way so it renders everything possible w/o network. You have 3 options:
This is the simplest one. Just open settings page from you user and click "Add to home screen". Note that PWA is only available from chrome and chrome android. No support for IOS and other browsers. But PWA is the most stable from ones below.
Use nativifier to create a client (replace pychat.org for your server): npx run nativifier pychat.org
cd frontend; yarn run electronProd
.You can use PWA as it's described in desktop app section which I recommend. Other way is cordova which is a lot harder. If you're not familiar with android SDK I would recommend doing the steps below from AndroidStudio:
"PUBLIC_PATH": "./"
, "BACKEND_ADDRESS": "pychat.org"
build into dist, rm .gz. , copy to www
. In index.html include <script src="cordova.js"></script>
bash download_content.sh android
Example for mac:
~/Library/Android/sdk/tools/bin/sdkmanager --licenses
brew install gradle
frontend/platforms/android
with androidStudioyarn start
; bash download_content.sh android
Debug
button should be available out of the box after openning a projectchrome://inspect/#devices
in chromeThe flow is the following
This section depends on the OS you use. I tested full install on Windows/Ubuntu/CentOs/MacOS/Archlinux/Archlinux(rpi3 armv7). pychat.org currently runs on Archlinux Raspberry Pi 3.
PATH
variable.pip
to install them.PATH
variable. Cygwin or git's will do find.(for example if you use only git PATH=C:\Program Files\Git\usr\bin;C:\Program Files\Git\bin
).cd backend
). Create virtualEnv python3 -m venv .venv
. Activate python virtual environment: source .venv/bin/activate
apt-get install python pip mysql-server libmysqlclient-dev
(python should be 3.6-3.8) If pip is missing check python-pip
. For old versions of Ubuntu you can use this ppa: sudo add-apt-repository ppa:deadsnakes/ppa; sudo apt-get update; sudo apt-get install python3.8 python3.8-dev python3.8-venv python3.8-apt; curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py; python3.8 get-pip.py
add-apt-repository -y ppa:rwky/redis; apt-get install -y redis-server
cd backend
). Create virtualEnv python3 -m venv .venv
. Activate python virtual environment: source .venv/bin/activate
pacman -S unzip python python-pip redis yarn mariadb python-mysqlclient
. nvm and pyenv is located in aur so yay -S nvm pyenv-virtualenv
(or use another aur package)mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
.cd backend; pyenv virtualenv 3.8-dev pychat
. Activate it source ~/.pyenv/versions/pychat/bin/activate
brew install mysql redis python3
brew services run mysql redis
cd backend
). Create virtualEnv python3 -m venv .venv
. Activate it: source .venv/bin/activate
Since we're using self singed certificate your OS doesn't know about for development. We need to do some tricks for browser to make it work. If you have valid certificates for your domain you can skip this step.
frontend/build/certs
directory. So you can skip this textcd frontend/build/certs
openssl genrsa -out private.key.pem 4096
openssl req -new -sha256 -out root.ca.pem -key private.key.pem -subj '/CN=localhost' -extensions EXT -config <( printf "[dn]\nCN=localhost\n[req]\ndistinguished_name = dn\n[EXT]\nsubjectAltName=DNS:localhost\nkeyUsage=digitalSignature\nextendedKeyUsage=serverAuth")
openssl x509 -req -days 3650 -in root.ca.pem -signkey private.key.pem -out server.crt.pem -extfile ./v3.ext
Useful links:
--ignore-certificate-errors
flag. E.g. on MacOS open -a Google\ Chrome --args --ignore-certificate-errors
Remember that Service Worker will work only if certificate is trusted. So flags like ignore-ceritifcate-errors won't work. But installing certifcate to root system will.excludeMAIN
file to .gitignore
or create link to exclude. ln -rsf .excludeMAIN .git/info/exclude
backend/chat/settings.py
. Modify file according to the comments in it.pip install -r requirements.txt
. (Remember you're still in backend
dir)echo "create database pychat CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci; CREATE USER 'pychat'@'localhost' identified by 'pypass'; GRANT ALL ON pychat.* TO 'pychat'@'localhost';" | mysql -u root
. You will need mysql running for that (e.g. systemctl start mysql
on archlinux) If you also need remote access do the same with '192.168.1.0/255.255.255.0';
bash ../download_content.sh create_django_tables
. (Remember you're still in backend
dir)Change to frontend directory cd frontend
I would recommend to use node version specified in nvm, so nvm install; nvm use
.
To get started install dependencies first: yarn install --frozen-lock
# or use npm if you're old and cranky
Take a look at copy development.json. The description is at Frontend config
vite dev-server is used for development purposes with hot reloading, every time you save the file it will automatically apply. This doesn't affect node running files, only watching files. yarn start
. If you open chrome and it's not loading, while main.ts is stuck at pending, increase system file descriptors. You can navigate to http://localhost:8080.
To build android use yarn run android -- 192.168.1.55
where 55 is your bridge ip address
To run electron use yarn run electronDev
. This will start electron dev. and generate /tmp/electron.html
and /tmp/electron.js
backend
as root directory for pycharm.backend
Settings:
to chat/settings.py
Settings
-> Project backend
-> Project Interpreter
-> Cogs in right top
-> 'Add' -> Virtual Environment
-> Existing environment
-> Interpereter
= pychatdir/.venv/bin/python
(or on Archlinux .pyenv/versions/pychat/bin/python
). Click ok. In previous menu on top 'Project interpreter` select the interpriter you just added.Settings
-> Project backend
-> Project structure
.idea
templates
directory as Template Folder
Run
-> Edit configuration
-> Django server
-> Checkbox Custom run command
start_tornado
. Remove port value.cd frotnend
Current linting supports:
<template>
<div>#[[$END$]]#</div>
</template>
<script lang="ts">
import {State} from '@/utils/store';
import {Component, Prop, Vue, Watch, Ref} from 'vue-property-decorator';
@Component
export default class ${COMPONENT_NAME} extends Vue {
}
</script>
<style lang="sass" scoped>
</style>
Disable tslint, since it's not used, and enable eslint:
Settings
Typescript
Tslint
Disable tslint
mysql
server if it's not started.redis-server
python manage.py start_tornado
Pychat is written in Python and typescript. For handling realtime messages WebSockets are used: browser support on client part and asynchronous framework Tornado on server part. For ORM django was used with MySql backend. Messages are being broadcast by means of redis pub/sub feature using tornado-redis backend. Redis is also used as django session backend and for storing current users online. For video call WebRTC technology was used with stun server to make a connection, which means you will always get the lowest ping and the best possible connection channel. Client part is written with progressive js framework VueJs which means that pychat is SPA, so even if user navigates across different pages websocket connection doesn't break. Pychat also supports OAuth2 login standard via FaceBook/Google. Css is compiled from sass. Server side can be run on any platform Windows, Linux, Mac. Client (users) can use Pychat from any browser with websocket support: IE11, Edge, Chrome, Firefox, Android, Opera, Safari...
Execute bash download_content.sh
it will show you help.
By default each user has turned off browser (console) logs. You can turn them on in /#/profile page (logs
checkbox). All logs are logged with window.logger
object, for ex: window.logger('message')()
. Note that logger returns a function which is binded to params, that kind of binding shows corrent lines in browser, especially it's handy when all source comes w/o libraries or other things that transpiles or overhead it. You can also inspect ws messages here for chromium. You can play with window.wsHandler.handleMessage(object)
and window.wsHandler.handle(string)
methods in debug with messages from log to see what's going on
Chat uses fontello and its api for icons. The decision is based on requirements for different icons that come from different fonts and ability to add custom assets. Thus the fonts should be generated (.wolf
etc). W/o this chat would need to download a lot of different fonts which would slow down the loading process. You can easily edit fonts via your browser, just execute bash download_content.sh post_fontello_conf
. Make your changes and hit "Save session". Then execute bash download_content.sh download_fontello
. If you did everything right new icons should appear under frontend/src/assets/demo.html
Server pings clients every PING_INTERVAL miliseconds. If client doesn't respond with pong in PING_CLOSE_JS_DELAY, server closes the connection. If ther're multiple tornado processes if can specify port for main process with MAIN_TORNADO_PROCESS_PORT. In turn the client expects to be pinged by the server, if client doesn't receive ping event it will close the connection as well. As well page has window listens for focus and sends ping event when it receives it, this is handy for situation when pc suspends from ram.
Pychat uses standard django migrations tools. So if you updated your branch from my repository and database has changed you need to ./manage.py makemigration
and ./manage.py migrate
. If automatic migration didn't work I also store migrations in migration. So you might take a look if required migration is there before executing commands. If you found required migration in my repo don't forget to change Migration.dependencies[]
and rename the file.
ScreenShare available for Chrome starting from v71. For chrome v31+ you should install an extension. It uses chrome.desktopCapture
feature that is available only via extension. The extension folder is located under screen_cast_extension`.
If you want to locally test it:
chrome://extensions/
url in chrome and verify that developer mode
checkbox is checked.load unpacked extension...
button and select screen_cast_extension directory.background.js
be able to receive messages from webpage you need to add your host to externally_connectable
section in manifest.jsonTp publish extension:
version
in manifest.json.bash download_content.sh zip_extension
extension.zip
to chrome webstore (Note, you need to have a developer account, that's 5$ worth atm).The successful connection produces logs below in console
Sender:
ws:in {"action": "offerCall", "content": {"browser": "Chrome 86"}, "userId": 2, "handler": "webrtc", "connId": "YZnbgKIL", "opponentWsId": "0002:UFBW", "roomId": 1, "time": 1604446797449}
WRTC Setting call status to received_offer
WRTC CallHandler initialized
ws:out {"action":"replyCall","connId":"YZnbgKIL","content":{"browser":"Chrome 86"},"messageId":1}
rsok33GN CallHandler initialized
rsok33GN:0005:EJAd Created CallSenderPeerConnection
WRTC Setting call status to accepted
WRTC capturing input
WRTC navigator.mediaDevices.getUserMedia({audio, video})
ws:out {"action":"acceptCall","connId":"YZnbgKIL","messageId":2}
YZnbgKIL:0002:UFBW Connect to remote
rsok33GN:0005:EJAd Creating RTCPeerConnection
YZnbgKIL:0002:UFBW Sending local stream to remote
rsok33GN:0005:EJAd Creating offer...
rsok33GN:0005:EJAd Created offer, setting local description
rsok33GN:0005:EJAd Sending offer to remote
YZnbgKIL:0002:UFBW onicecandidate
...
YZnbgKIL:0002:UFBW onicecandidate
rsok33GN:0005:EJAd onsendRtcData
rsok33GN:0005:EJAd answer received
rsok33GN:0005:EJAd onaddstream
rsok33GN:0005:EJAd onsendRtcData
Receiver:
WRTC capturing input
WRTC navigator.mediaDevices.getUserMedia({audio, video})
WRTC got local stream MediaStream {id: "0IeyYT9LxHRidUZaw7XSVnXEPWYimm4KDmJB", active: true, onaddtrack: null, onremovetrack: null, onactive: null, …}
WRTC Setting call status to sent_offer
ws:out {"action":"offerCall","roomId":1,"content":{"browser":"Chrome 86"},"messageId":1}
ws:in {"action": "setConnectionId", "handler": "void", "connId": "YZnbgKIL", "messageId": 1, "time": 1604446797449}
rsok33GN CallHandler initialized
rsok33GN:0004:oIc5 Created CallReceiverPeerConnection
YZnbgKIL:0001:qobF Connect to remote
rsok33GN:0004:oIc5 Creating RTCPeerConnection
YZnbgKIL:0001:qobF Sending local stream to remote
rsok33GN:0004:oIc5 onsendRtcData
rsok33GN:0004:oIc5 Creating answer
rsok33GN:0004:oIc5 onaddstream
rsok33GN:0004:oIc5 Sending answer
rsok33GN:0004:oIc5 onsendRtcData
rsok33GN:0004:oIc5 onsendRtcData
The string rsok33GN:0005:EJAd
describes:
rsok33GN
is ID of CallHandler0005
is Id of userEJAd
id of connection (TornadoHandler.id
)TO see current connections and their info check chrome://webrtc-internals/ Read this article to understand how JSEP architecture works. SEE WEBRTC_CONFIG at development.json. I personally use turn server coturn, It needs ports 3478 to be exposed.
The technologies stack used in project:
builder.js is used to build project. Take a look at it to understand how source files are being processed. Its start point is entry: ['./src/main.ts']
. Everything is imported in this files are being processed by section loaders
.
Every vue component has injected .$logger
object, to log something to console use this.logger.log('Hello {}', {1:'world'})();
Note calling function again in the end. Logger is disabled for production. For more info visit lines-logger
This project uses vue-property-decorator (that's has a dependency vue-class-component) vuex-module-decorators. You should write your component as the following:
import { Vue, Component, Prop, Watch, Emit, Ref } from 'vue-property-decorator'
import {userModule, State} from '@/utils/storeHolder'; // vuex module example
@Component
export class MyComp extends Vue {
@Ref
button: HTMLInputElement;
@Prop readonly propA!: number;
@State
public readonly users!: User[];
@Watch('child')
onChildChanged(val: string, oldVal: string) { }
@Emit()
changedProps() {}
async created() {
userModule.setUsers(await this.$api.getUsers());
}
}
development.json and production.json have the following format:
{
"BACKEND_ADDRESS": "e.g. pychat.org:443, protocol shouldn't be there, note there's no trailing slash, you can specify '{}' to use the same host as files served with",
"IS_DEBUG": "if true, build won't be uglifies, logs will be set to trace, window object will be added with useful data and etc",
"GOOGLE_OAUTH_2_CLIENT_ID" : "check chat/settings_example.py",
"FACEBOOK_APP_ID": "check chat/settings_example.py",
"RECAPTCHA_PUBLIC_KEY": "check chat/settings_example.py RECAPTCHA_SITE_KEY",
"AUTO_REGISTRATION": "if set to true, for non loggined user registration page will be skipped with loggining with random generated username. Don't use RECAPTCHA with this key",
"PUBLIC_PATH": "Set this path if you have different domains/IPs for index.html and other static assets, e.g. I serve index.html directly from my server and all sttatic assets like main.js from CDN, so in my case it's 'https://static.pychat.org/' note ending slash",
"ISSUES": "if true navigation bar will display link to reporting a issue page",
"GIPHY_API_KEY": "Api keys that is used to fetch gifs from https://giphy.com/. Be aware, this key is gonna be exposed to frontend. So anyone can steal it. To get those sign up in https://developers.giphy.com/, create a new app and replaced with its key.",
"GITHUB_LINK": "an external link to project source files, in my case https://github.com/Deathangel908/pychat . Set to false if you don't wanna see it in the navbar",
"FLAGS": "if true, a user name will contain a country icon on the right. User names are shown on the right section of the screen",
"WEBRTC_CONFIG": "This variable defines the first argument of RtcPeerConnection constructor. Sometimes webrtc stun server doesn't work in establishing a connection. Especially for this you can use turn server instead of it. Docker prod docker image already comes with a turn server, example of configuration for it `{iceServers:[{urls:['turn:YOUR_DOMAIN'],username:'pychat',credential:'pypass'}]}`. replace YOUR_DOMAIN with your real domain name/public ip. You other scenarios use your server like coturn (https://github.com/coturn/coturn). See more info of this variable at docs: https://developer.mozilla.org/en-US/docs/Web/API/RTCPeerConnection/RTCPeerConnection#RTCConfiguration_dictionary"
}
In order to setup continuous delivery via github:
mkdir /tmp/sshkey; ssh-keygen -t rsa -b 4096 -C "github actions" -f /tmp/sshkey/id_rsa
/tmp/sshkey/id_rsa.pub
to server ~/.ssh/authorized_keys
where ~
is the home for ssh user to use ( I used http
)HOST
-ssh host (your domain)PORT
- ssh port (22)SSH_USER
- ssh user, if you used my setup it's http
ID_RSA
- what ssh-keygen has generated in step above to/tmp/sshkey/id_rsa
/etc/sudoers
withCmnd_Alias RESTART_TORNADO = /usr/bin/systemctl restart tornado
http ALL=(ALL) NOPASSWD: RESTART_TORNADO