Commit 90c5384f authored by Ilya Rassadin's avatar Ilya Rassadin
Browse files

Improve DOC_ROOT_NAME feature with rsync-filter-support 

<h3>The problem</h3>

The CMS-specific filter files (rsync-filter-bitrix, rsync-filter-ceteracms, rsync-filter-wordpress) all hardcode /www/ in their rules. rsync-filter-std operates at the project root level and has no /www/ references, so it needs no changes.

When DOC_ROOT_NAME=public_html, the deployment renames `www/` to `public_html/` in the checkout, then rsyncs from $CI_PROJECT_DIR/ to the server. But the filter still says exclude `/www/**` — which now matches nothing (the dir is public_html/), so rsync
happily deletes everything inside public_html/ on the server that isn't in the repo.

<h3>The solution</h3>

Runtime sed substitution after filter files are fetched — replace `/www/` with `/$DOC_ROOT_NAME/` in the downloaded filter file.

Where exactly: in `.deploy_code_to_server_with_delete`, right after `*set_rsync_filter` (which sets `$PROJECT_FILTER`). At that point both filter files are on disk and DOC_ROOT_NAME is available from the dotenv artifact.

```bash
if [[ "$DOC_ROOT_NAME" != "www" ]]; then
sed -i "s|/www/|/$DOC_ROOT_NAME/|g" "$PROJECT_FILTER"
fi
```

Only `$PROJECT_FILTER` needs it — `rsync-filter-std` has no `/www/` paths.

* Why not put it inside the `*set_rsync_filter anchor`?

  The anchor doesn't know about `rsync-filter-std`, and keeping it clean/single-purpose is better. The substitution is a concern of the deploy job, not of filter selection.
* Why not modify the filter files themselves to use a variable?

  rsync filter files are plain text — they don't support shell variable interpolation. The only way to parametrize them is runtime substitution (sed/envsubst) or generating them on the fly. Building templating system for them looks too complex for such a simple thing.
* What about projects supplying their own local rsync-filter?

  Those also need the substitution, and `*set_rsync_filter` correctly sets `PROJECT_FILTER="rsync-filter"` for them — so the same one sed line covers that case too.
parent ae25008c

.build_composer.yml

+5 −3
Original line number Diff line number Diff line
@@ -35,8 +35,10 @@ 
      exit(1);

  }

  function replace_www($path, $docRootName) {

      if (strpos($path, 'www/') === 0) {

          return $docRootName . '/' . substr($path, 4);

      $defaultDocRootName = getenv('DEFAULT_DOC_ROOT_NAME');

      $prefix = $defaultDocRootName . '/';

      if (strpos($path, $prefix) === 0) {

          return $docRootName . '/' . substr($path, strlen($prefix));

      }

      return $path;

  }
@@ -54,7 +56,7 @@ 
  file_put_contents($composerFile, $newJson);

  echo "composer.json updated successfully.\n";

  PHP

  if [ -e $COMPOSER_INSTALL_DIR/composer.json ] && [ ! -z "$DOC_ROOT_NAME" ] && [ "$DOC_ROOT_NAME" != "www" ]; then

  if [ -e $COMPOSER_INSTALL_DIR/composer.json ] && [ ! -z "$DOC_ROOT_NAME" ] && [ "$DOC_ROOT_NAME" != "$DEFAULT_DOC_ROOT_NAME" ]; then

    php fix_composer_autoload.php

    echo "autoload has been adjusted to use $DOC_ROOT_NAME instead of www"

  else

.deploy_code_to_server.yml

+13 −9
Original line number Diff line number Diff line
@@ -5,14 +5,14 @@ 
    group: www-data

  script:

    - |

      # Proceed only if DOC_ROOT_NAME is set, non-empty, and not "www"

      #  and www directory exists in current working directory

      # Proceed only if DOC_ROOT_NAME is set, non-empty, and differs from DEFAULT_DOC_ROOT_NAME

      #  and $DEFAULT_DOC_ROOT_NAME directory exists in current working directory

      #  and $DOC_ROOT_NAME directory not exists

      if [[ -n "${DOC_ROOT_NAME+x}" && -n "$DOC_ROOT_NAME" && "$DOC_ROOT_NAME" != "www" && -d "www" && ! -e "$DOC_ROOT_NAME" ]]; then

          mv www "$DOC_ROOT_NAME"

      if [[ -n "${DOC_ROOT_NAME+x}" && -n "$DOC_ROOT_NAME" && "$DOC_ROOT_NAME" != "$DEFAULT_DOC_ROOT_NAME" && -d "$DEFAULT_DOC_ROOT_NAME" && ! -e "$DOC_ROOT_NAME" ]]; then

          mv "$DEFAULT_DOC_ROOT_NAME" "$DOC_ROOT_NAME"

          echo "renamed www to $DOC_ROOT_NAME"

      fi

    - rsync --links --safe-links -avz --chown=$user:$group -e "ssh $SSH_OPTS" $CI_PROJECT_DIR/www/ $DEPLOY_HOST_PATH/www/

    - rsync --links --safe-links -avz --chown=$user:$group -e "ssh $SSH_OPTS" $CI_PROJECT_DIR/$DOC_ROOT_NAME/ $DEPLOY_HOST_PATH/$DOC_ROOT_NAME/

  dependencies:

    - setup_environment_vars

  only:
@@ -44,13 +44,17 @@ 
    - 'which curl || (apt-get update -y && apt-get install curl -y)'

    - curl -O https://gitlab.cetera.ru/boilerplate/ci/raw/master/rsync-filter-std

    - *set_rsync_filter

    - |

      if [[ "$DOC_ROOT_NAME" != "$DEFAULT_DOC_ROOT_NAME" ]]; then

          sed -i "s|/www/|/$DOC_ROOT_NAME/|g" "$PROJECT_FILTER"

      fi

    - echo "PROJECT_FILTER set to $PROJECT_FILTER; user set to $user; group set to $group"

    - |

      # Proceed only if DOC_ROOT_NAME is set, non-empty, and not "www"

      #  and www directory exists in current working directory

      # Proceed only if DOC_ROOT_NAME is set, non-empty, and differs from DEFAULT_DOC_ROOT_NAME

      #  and $DEFAULT_DOC_ROOT_NAME directory exists in current working directory

      #  and $DOC_ROOT_NAME directory not exists

      if [[ -n "${DOC_ROOT_NAME+x}" && -n "$DOC_ROOT_NAME" && "$DOC_ROOT_NAME" != "www" && -d "www" && ! -e "$DOC_ROOT_NAME" ]]; then

          mv www "$DOC_ROOT_NAME"

      if [[ -n "${DOC_ROOT_NAME+x}" && -n "$DOC_ROOT_NAME" && "$DOC_ROOT_NAME" != "$DEFAULT_DOC_ROOT_NAME" && -d "$DEFAULT_DOC_ROOT_NAME" && ! -e "$DOC_ROOT_NAME" ]]; then

          mv "$DEFAULT_DOC_ROOT_NAME" "$DOC_ROOT_NAME"

          echo "renamed www to $DOC_ROOT_NAME"

      fi

    - rsync --links --safe-links -avz --chown=$user:$group -e "ssh $SSH_OPTS" --filter="merge rsync-filter-std" --filter="merge $PROJECT_FILTER" $RSYNC_PARAMS $CI_PROJECT_DIR/ $DEPLOY_HOST_PATH/

CLAUDE.md

0 → 100644
+102 −0
Original line number Diff line number Diff line 
# CI/CD Boilerplate Repository



This is the central CI/CD template library for Cetera web projects. Projects include these files directly from GitLab via raw URLs and extend the job templates.



## Repository Purpose



Reusable GitLab CI/CD job definitions and supporting files consumed by project-level `.gitlab-ci.yml`. Projects include specific `.yml` files from this repo and extend the hidden jobs (prefixed with `.`).



## File Structure



### Pipeline stages (`stages.yml`)

Fixed order: `prebuild → test → build → post_build → deploy → post_deploy`



### `before_script.yml` — Core setup

The foundation everything else builds on. Defines:

- **`before_script`** (global): installs ssh/rsync/perl, sources `.env.project`

- **`.set_path_vars`** anchor: computes `PROJECT_SERVER`, `DEPLOY_HOST_PATH`, `PROJECT_ROOT`, `PROJECT_PORT`, etc. from project env vars

- **`.set_ssh_keys`** anchor: decodes base64 `DEPLOY_KEY_PROD` into `~/.ssh/id_rsa`

- **`setup_environment_vars`** job (prebuild stage): runs only on protected branches, exports all path vars

- **`.before_deploy_code`**: base for all deploy jobs — sets SSH config, loads keys and path vars

- **workflow rules**: `ENVIRONMENT_NAME=staging` for beta branch, `production` for master



### Job template files



| File | Hidden job(s) | Stage | Purpose |

|------|--------------|-------|---------|

| `.build_assets.yml` | `.build_assets` | build | `npm install && npm run minify` for each layout in `working/` |

| `.build_composer.yml` | `.build_composer`, `.build_composer_dev` | build/prebuild | `composer install` with audit and autoload tuning |

| `.deploy_assets_to_staging.yml` | `.deploy_assets_to_staging` | deploy | rsync built assets to `wireframes.cetera.ru` |

| `.deploy_code_to_server.yml` | `.deploy_code_to_server`, `.deploy_code_to_server_with_delete` | deploy | rsync `www/` to production/staging server |

| `.deploy_composer_files_to_server.yml` | `.deploy_composer_files_to_server` | deploy | rsync vendor/ artifacts to server |

| `test.yml` | `.lint_js`, `.phpunit`, `.phpcs_test`, `.pages` | test | JS lint, PHPUnit coverage, PHP_CodeSniffer, Pages deploy |



### Rsync filter files



Selected at deploy time by `.set_rsync_filter` anchor based on `PROJECT_CMS_TYPE`:



| File | CMS | What it preserves on server |

|------|-----|-----------------------------|

| `rsync-filter-bitrix` | Bitrix | `/www/bitrix/***`, `/www/upload/***` |

| `rsync-filter-ceteracms` | CeteraCMS | `/www/cms/***`, `/www/library/***`, `/www/plugins/***`, `/www/imagetransform/***` |

| `rsync-filter-wordpress` | WordPress | wp-admin, wp-includes, plugins, uploads |

| `rsync-filter-react` | React | (empty — no server-side files to protect) |

| `rsync-filter-std` | default/Laravel | general excludes (git, working/, boilerplate/, vendor/, etc.) |



If a project provides its own `rsync-filter` file, that takes precedence over the CMS-specific one.



### `.settings_extra.php`

Bitrix-specific PHP config deployed to `$DOC_ROOT_NAME/bitrix/` during deploy. Switches cache/session/DB backends based on `RUN_MODE` env var:

- (default/production): Memcached + Redis on localhost

- `staging`: same as production but analytics disabled

- `development`: Memcache + DB pointing to docker services (`memcached:11211`, localhost cetera:cetera), debug on

- `load_testing`: separate DB credentials



### `gitlab-ci-template.yml`

Example template showing the typical include block and commented-out job extensions a project would use.



## How Projects Use This Repo



Projects include raw URLs of the files they need and extend the hidden jobs:



```yaml

include:

  - 'https://gitlab.cetera.ru/boilerplate/ci/raw/master/before_script.yml'

  - 'https://gitlab.cetera.ru/boilerplate/ci/raw/master/stages.yml'

  - 'https://gitlab.cetera.ru/boilerplate/ci/raw/master/.deploy_code_to_server.yml'



deploy_code_to_server_with_delete:

  extends: .deploy_code_to_server_with_delete

  variables:

    RSYNC_PARAMS: '--delete --dry-run'

  only:

    refs:

      - master

      - beta

```



## Key Environment Variables (set in project CI settings)



| Variable | Purpose |

|----------|---------|

| `PROJECT_SERVER` / `PROJECT_SERVER_BETA` | Target server hostname |

| `DEPLOY_KEY_PROD` | Base64-encoded SSH private key |

| `PROJECT_CMS_TYPE` | Selects rsync filter (`bitrix`, `ceteracms`, `wordpress`, `react`) |

| `PROJECT_SITE` | Site/domain name used in server path |

| `DOC_ROOT_NAME` | Document root folder name (default: `www`) |

| `SSH_USER` | SSH user on target server (default: `root`) |

| `RSYNC_PARAMS` | Extra rsync flags (e.g. `--delete`, `--dry-run`) |

| `SKIP_SETTINGS_EXTRA_DEPLOYMENT` | Skip `.settings_extra.php` deploy for Bitrix |



## Branching Model



- **master** → `ENVIRONMENT_NAME=production`, deploys to production server

- **beta** → `ENVIRONMENT_NAME=staging`, deploys to staging server (`PROJECT_SERVER_BETA`)

- Protected branches trigger `setup_environment_vars` in prebuild stage



## Adding or Modifying Jobs



- Hidden jobs (`.job_name`) are templates — never run directly

- Anchors (`&anchor_name`) are YAML anchors merged with `<<: *anchor_name` — not GitLab extends

- `extends:` pulls in the GitLab job template; `variables:` in the project job override template defaults

- Changes here affect all projects that include from master — test carefully

before_script.yml

+4 −1
Original line number Diff line number Diff line
@@ -91,9 +91,12 @@ before_script: 
  export SKIP_SETTINGS_EXTRA_DEPLOYMENT=${SKIP_SETTINGS_EXTRA_DEPLOYMENT:-"false"}

  echo "SKIP_SETTINGS_EXTRA_DEPLOYMENT=$SKIP_SETTINGS_EXTRA_DEPLOYMENT" >> cetera_build.env



  export DEFAULT_DOC_ROOT_NAME="www"

  echo "DEFAULT_DOC_ROOT_NAME=$DEFAULT_DOC_ROOT_NAME" >> cetera_build.env



  DOC_ROOT_NAME_VAR="${CETERA_VAR_PREFIX}_DOC_ROOT_NAME"

  DOC_ROOT_NAME=${!DOC_ROOT_NAME_VAR}

  export DOC_ROOT_NAME=${DOC_ROOT_NAME:-"www"}

  export DOC_ROOT_NAME=${DOC_ROOT_NAME:-"$DEFAULT_DOC_ROOT_NAME"}

  echo "DOC_ROOT_NAME=$DOC_ROOT_NAME" >> cetera_build.env



  USER_VAR="${CETERA_VAR_PREFIX}_USER"