Storage Backing for BlockStructureStore

When the user access the course, there is a request trying to access BlockStructure (BS) data in Memcache and when it is not found in the cache, the system enters Collect Mode, and the process is quite heavy and causes slower response times.

The maximum size of Memcache is 1 MB so if the BS is bigger than that Memcache will not save it.

The solution to the BS issue against Memcache is to enable storage backing, be it local storage or Amazon S3 storage.

1. Go to Open edX server console.

2. Open and edit file /edx/app/edxapp/lms.env.json and /edx/app/edxapp/cms.env.json.

3. Then BLOCK_STRUCTURES_SETTINGS:

   Use local storage

   1. Set PRUNING_ACTIVE to true (default: false).

   2. Add DIRECTORY_PREFIX and set it with your desired directory name. For example use BLOCK_STRUCTURES.

      ...
          "BLOCK_STRUCTURES_SETTINGS": {
              "COURSE_PUBLISH_TASK_DELAY": 30, 
              "PRUNING_ACTIVE": true, 
              "TASK_DEFAULT_RETRY_DELAY": 30, 
              "TASK_MAX_RETRIES": 5,
              "DIRECTORY_PREFIX": "BLOCK_STRUCTURES/"
          }, 
      ...
      

   Use Amazon S3

   1. Set PRUNING_ACTIVE to true (default: false).

   2. Add DIRECTORY_PREFIX and set it with your desired directory name. For example use BLOCK_STRUCTURES.

   3. Add STORAGE_CLASS, STORAGE_KWARGS as in the following example

      ...
          "BLOCK_STRUCTURES_SETTINGS": {
              "COURSE_PUBLISH_TASK_DELAY": 30,
              "PRUNING_ACTIVE": true,
              "TASK_DEFAULT_RETRY_DELAY": 30,
              "TASK_MAX_RETRIES": 5,
              "STORAGE_CLASS": "storages.backends.s3boto.S3BotoStorage",
              "STORAGE_KWARGS": {
                  "AWS_STORAGE_BUCKET_NAME": "pacifictraining-course"
              },
              "DIRECTORY_PREFIX": "/BLOCK_STRUCTURES/"
          }, 
      ...
      

4. Create and save BS in S3 bucket or local storage you are using.

   ./manage.py lms generate_course_blocks \
   --all_courses \
   --with_storage \
   --force_update
   

5. Enable the following switches from the admin page (e.g https://edx.sibunglon.com/admin/waffle/switch/).

   • block_structure.storage_backing_for_cache
   • block_structure.raise_error_when_not_found

6. Restart lms, cms, dan edxapp_worker services.