October 25, 2020

Log Open edX Not Update

Activities, Android Developer, Ansible, Apache2, Atlassian, Ayo Belajar Linux, Bestpath Network, BLC Telkom Klaten, BSD, Caddy Server, Case Study, Cisco, Cisco Indonesia, Cloud Computing, Cockpit, Custom Weapons, Docker, E-Learning, Engenius, Error, FreeBSD, FreeBSD Indonesia, Komunitas Pengguna Linux Indonesia, KPLI Bulukumba, KPLI Klaten, Lets Encrypt, Linux, MacOS, Microsoft Azure, Microsoft SQL Server, MikroTik, MikroTik Indonesia, MySQL, Nginx, Open edX, OpenSID, Others, PHP, phpMyAdmin, PostgreSQL, Proxmox, Python, Redash, Sendy, SSH, Stories, Subnetting, TP-Link, Ubiquiti, Unix, Virtualization, Windows, X-Mosque, Faizar Septiawan, Icar, siBunglonGanteng, Orang Ganteng, siBunglonLabs, Programmer, SysAdmin, Site Reliability Engineer, Developer, Palugada, Makassar, Ganteng, Gila, Cyclist, Panglima, Setan, Panglima Setan, sibunglon, Ganteng

Case

I’ve attach a new disk on my Open edX server, I move Open edX log file into a new disk and create symbolic link to /edx/var because I don’t want to make a lot of changes in configuration or hardcode of Open edX.

I moved the log files to the new disk, I did the following:

$ # Stop services
$ /edx/bin/supervisorctl stop all
$ sudo systemctl stop mongod rabbitmq-server supervisor nginx

$ # Rsync log directory to the new disk and make backup on local storage server
$ sudo rsync -avP /edx/var/log /edx/backup
$ sudo rsync -avP /edx/var/log /media-edx

$ # Remove log from edxapp
$ sudo rm -rf /edx/var/log

$ # Create symbolic link
$ sudo ln -sf /media-edx/log /edx/var

$ # Start services
$ sudo systemctl start mongod rabbitmq-server supervisor nginx

When services edX has been start, log file for lms, cms, and tracking is not update.

Solution

I’m run this command on Open edX server.

$ sudo lsof -nn -p $(pidof rsyslogd)

And find this.

---
rsyslogd 2502 syslog    8w   REG 202,1   104721    1824885 /edx/var/log/ecomworker/edx.log (deleted)
rsyslogd 2502 syslog    9w   REG 202,1    34915    1827893 /edx/var/log/xqueue/edx.log (deleted)
rsyslogd 2502 syslog   10w   REG 202,1    58789    1824922 /edx/var/log/lms/edx.log (deleted)
rsyslogd 2502 syslog   11w   REG 202,1     3821    1824921 /edx/var/log/insights/edx.log (deleted)
rsyslogd 2502 syslog   12w   REG 202,1   126444    1824869 /edx/var/log/cms/edx.log (deleted)
rsyslogd 2502 syslog   13w   REG 202,1    27029    1827788 /edx/var/log/tracking/tracking.log (deleted)
---

Restart rsyslog for reload file log path.

$ sudo systemctl restart rsyslog

When check list of open files using this command.

$ sudo lsof -nn -p $(pidof rsyslogd)

I find this.

---
rsyslogd 9756 syslog    8w   REG 0,40   111284  4805268258894419596 /media-edx/log/ecomworker/edx.log
rsyslogd 9756 syslog    9w   REG 0,40    45721 13793588914623709624 /media-edx/log/xqueue/edx.log
rsyslogd 9756 syslog   10w   REG 0,40   130154 17067885752292221460 /media-edx/log/cms/edx.log
rsyslogd 9756 syslog   11w   REG 0,40    77549 11746508073636907799 /media-edx/log/lms/edx.log
rsyslogd 9756 syslog   12w   REG 0,40    29150   767045182461436438 /media-edx/log/tracking/tracking.log
rsyslogd 9756 syslog   13w   REG 0,40     4155  5614057283095338575 /media-edx/log/insights/edx.log
---

And when i re-check log file for lms, cms, and tracking. Log file is back to normal, log file is up to update.