Syncing from GitLab to Docker Host Volume

This week I was asked to help solve a problem encountered during a POC.

The team were working with an application in a docker container which needs to read configuration data on startup.

Rather than having the configuration data baked into the container, the application reads it from a docker data volume which is mapped to a directory on the host.

The configuration data was copied to the host, the volumes and directories mapped and the application tested successfully.

The configuration data files are maintained in a git repository, and for the first test these had just been copied to the host.  However, the team were asked if we could keep the configuration data in sync with the git repository, so we'd have something like:


Not something I had done before.  However I was fairly sure that I wasn't the first.


  • I had a quick search on dockerhub and found an image that looked useful:  https://hub.docker.com/r/openweb/git-sync/
  • Basically it polls a git repo and clones it to a directory in the container that is mapped to a docker volume, and so to a directory on the host.
  • I tested it against a public repo on github and checked that it not only cloned the repo, but also synchronised subsequent changes.
  • Then I pointed to the URL for the config repo.
    • I had to include the username and password in the URL
    • I had to set --network==host to use the SSH tunnel between the host and the GitLab server.
    • I also needed to set the environment variable GIT_SSL_NO_VERIFY=true, otherwise it fails with an SSL verification error
  • Issued the docker run command and it worked :-)
Of course if you wanted to get fancy you could use WebHooks from GitLab to trigger a "clone on commit", and the networking and security could probably be improved upon, but it did the job.

Comments

Post a Comment

Popular posts from this blog

Wot no FDK?

Writing Serverless Functions in a New Language Background Fn Project  is an open - source, Docker based, cloud native serverless platform .  You can run it in any environment that supports Docker.  Oracle Functions is based on Fn Project. If you want to have a play with Fn before reading the rest of this post, I recommend the " Quick Start " guide. One of the great strengths of Fn Project is that it is open. When other serverless platforms (that shall remain nameless) restricted you to JavaScript / Node plus maybe a couple of others, Fn offered an FDK for Ruby (for those who prefer elegance and beauty in their code). This was a major factor in me getting involved with the project, and becoming a contributor to the Ruby FDK. In addition to Ruby, Fn offers a number of other FDKs: Go, Java, Python, Node and .NET Core. But beyond that, Fn has always been open to additional language runtimes.  If your code can run in a Docker container, then (in principle) it can run as an Fn fun
Read more

The Case of the Vanishing Dockerfile

If your serverless platform is Oracle Functions or the open source  Fn Project that it is based on, you may have wondered exactly how a function image gets built for one of the standard runtimes. One clue is that if you're watching closely you may see a Dockerfile  a p pear briefly, then vanish during the build process. The Fn CLI will create the Dockerfile for the runtime, build the function and then delete it once the build has completed. You can see the steps of the docker build by running  fn build  or  fn deploy  with the  --verbose  flag specified: $ fn build --verbose [14:57:34] Building image fra.ocir.io/oraseemeatechse/crush157/dummy:0.0.1 FN_REGISTRY: fra.ocir.io/oraseemeatechse/crush157 Current Context: default Sending build context to Docker daemon 14.34kB Step 1/11 : FROM fnproject/fn-java-fdk-build:jdk11-1.0.108 as build-stage ---> e3c33b854f29 Step 2/11 : WORKDIR /function ---> Using cache ---> c0bcff391281
Read more

Honey I Shrunk The Container

Image
(or how I found out about microcontainers and why I think you should use them) Context About 18 months ago I was asked to help an ISV partner who wanted to migrate their product to cloud. At the time they had a very traditional software industry business model - sell the customer a license to run their software, along with a subscription for ongoing support. However they wanted to be able to move to a SaaS model where the customer would consume their software as a service running in the cloud. Their stack consisted of: a Ruby on Rails application Postgres Redis Their initial attempt at creating a SaaS offering had been to run this as a set of VMs on top of IaaS. While the software ran just fine, they soon saw that this approach wasn't going to give them a successful SaaS business. If any part of the application were experiencing high demand, the only way to scale it was to start another instance of the whole application . Since they were packaging the applic
Read more