{"id":69,"date":"2019-11-19T19:02:51","date_gmt":"2019-11-19T18:02:51","guid":{"rendered":"https:\/\/blog.nubisoft.pl\/?p=69"},"modified":"2021-12-11T07:19:50","modified_gmt":"2021-12-11T06:19:50","slug":"private-file-sharing-using-nextcloud-with-wasabi-s3-a-good-dropbox-replacement-part-1","status":"publish","type":"post","link":"https:\/\/nubisoft.io\/blog\/private-file-sharing-using-nextcloud-with-wasabi-s3-a-good-dropbox-replacement-part-1\/","title":{"rendered":"Private file sharing using NextCloud with Wasabi S3 &#8211; a good dropbox replacement (Part1)"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\">In this post, we present how to deploy and configure NextCloud &#8211; the data-sharing platform &#8211; to store and secure files in an efficient way in terms of security and cost optimization.<\/h2>\n\n\n\n<p>Nowadays, data-sharing among team members is a must. While in the case of source code there is practically one single way &#8211; Git (or to be more specific two ways: github.com and gitlab.com), in the case of file-sharing in general, there are many different ways. Probably we all love the dropbox platform with its awesome usability\/simplicity factor. However many of you are pointing two major drawbacks:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>After exceeding a certain point of data volume the DropBox services are getting noticeably more expensive.<\/li><li>All the shared data are stored at the side of the DropBox service provider, so to be honest we don&#8217;t know who else gets into our data and what does it do with it.<\/li><\/ol>\n\n\n\n<p>There are many tips all around WWW regarding DropBox substitution and setting up some open source solutions on private VPS. But to be honest &#8211; until you try, you won&#8217;t realize how incomplete they are, and how challenging this problem is.<\/p>\n\n\n\n<p>To resolve the problem stated above on our own, we gave a try to ownCloud platform. After several preliminary setting up, we acquired some experiences that let us make the final deployment. Meanwhile, we moved toward a sibling project <a href=\"https:\/\/nextcloud.com\/\">NextCloud<\/a>, but we have to emphasize that this decision was not driven by any painful experiences, but rather by our observations of great community support of NextCloud. Probably, using ownCloud you can be successful to the same extent as we are with NextCloud.<\/p>\n\n\n\n<p>We decided to deploy NextCloud to the cheapest <a href=\"https:\/\/www.digitalocean.com\/\">DigitalOcean <\/a>droplet keeping hope that computing resources will be sufficient to support our team of several dozen people. To simplify the installation and maintenance of the whole configuration, we&#8217;ve used containerization technologies. There are plenty of <strong>docker-compose.yml<\/strong> templates on the Internet, but none of them appeared as an out of box solution for our needs, so we had to develop one on our own.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>version: '3'\n\nservices:\n\n  redis:\n    image: redis:5-alpine\n    restart: always\n\n  db:\n    image: postgres:11-alpine\n    restart: always\n    volumes:\n      - db:\/var\/lib\/postgresql\/data\n    env_file:\n      - db.env\n\n  app:\n    image: nextcloud:fpm-alpine\n    restart: always\n    volumes:\n      - nextcloud:\/var\/www\/html\n    environment:\n      - POSTGRES_HOST=db\n      - POSTGRES_DB=nextcloud\n      - REDIS_HOST=redis\n    env_file:\n      - db.env\n    depends_on:\n      - db\n      - redis\n\n  web:\n    build: .\/web\n    restart: always\n    volumes:\n      - nextcloud:\/var\/www\/html:ro\n    environment:\n      - VIRTUAL_HOST=x.y.com\n      - LETSENCRYPT_HOST=x.y.com\n      - LETSENCRYPT_EMAIL=x@y.com\n    depends_on:\n      - app\n    networks:\n      - proxy-tier\n      - default\n\n  proxy:\n    build: .\/proxy\n    restart: always\n    ports:\n      - 80:80\n      - 443:443\n    labels:\n      com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: \"true\"\n    volumes:\n      - certs:\/etc\/nginx\/certs:ro\n      - vhost.d:\/etc\/nginx\/vhost.d\n      - html:\/usr\/share\/nginx\/html\n      - \/var\/run\/docker.sock:\/tmp\/docker.sock:ro\n    networks:\n      - proxy-tier\n\n  letsencrypt-companion:\n    image: jrcs\/letsencrypt-nginx-proxy-companion\n    restart: always\n    volumes:\n      - certs:\/etc\/nginx\/certs\n      - vhost.d:\/etc\/nginx\/vhost.d\n      - html:\/usr\/share\/nginx\/html\n      - \/var\/run\/docker.sock:\/var\/run\/docker.sock:ro\n    networks:\n      - proxy-tier\n    depends_on:\n      - proxy\n\nvolumes:\n  db:\n  nextcloud:\n  certs:\n  vhost.d:\n  html:\n\nnetworks:\n  proxy-tier:<\/code><\/pre>\n\n\n\n<p>Let us dive into details!<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Databases<\/li><\/ol>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"841\" height=\"775\" src=\"https:\/\/blog.nubisoft.pl\/wp-content\/uploads\/2019\/11\/nc-db.png\" alt=\"\" class=\"wp-image-73\" srcset=\"https:\/\/nubisoft.io\/blog\/wp-content\/uploads\/2019\/11\/nc-db.png 841w, https:\/\/nubisoft.io\/blog\/wp-content\/uploads\/2019\/11\/nc-db-300x276.png 300w, https:\/\/nubisoft.io\/blog\/wp-content\/uploads\/2019\/11\/nc-db-768x708.png 768w\" sizes=\"auto, (max-width: 841px) 100vw, 841px\" \/><\/figure>\n\n\n\n<p>1a. Redis<\/p>\n\n\n\n<p>If you want to go with your NextCloud to production, never go without an in-memory database. Not using it will result in frequent errors of file locking (plenty of messages: <code>server replied: &lt;file&gt; is locked<\/code>) at the clients and overall performance degradation of the server. We&#8217;ve used Redis and it appears to perform very well.<\/p>\n\n\n\n<p>And yes, you may follow tips published on the Internet to switch  file locking off using the option <code>'filelocking.enabled' =&gt; false<\/code>.  Good luck to you, Suicide! \ud83d\ude09 <\/p>\n\n\n\n<p>1b. PostgreSQL<\/p>\n\n\n\n<p>As a persistent data store, we choose PostgreSQL as we are excited about its directions and velocity of development. We claim that no much time left (if any) to catching up with enterprise-class competitors such as Oracle or MsSQL. Two details deserve additional attention. First, we configured data directory as external (volumes option), which means that regardless of the PostgreSQL container life cycle, the data remain persistent. Second, we enforced the major version of PostgreSQL (as 11), despite many docker-compose configurations found on WWW just skip it. Justification for this is that by upgrading containers (<code>docker pull<\/code> command) containers with the highest versions are downloaded. While in the case of the NextCloud container the developers have foreseen automation of migration, then in the case of the PostgreSQL container you have to take care of it by yourself. An example of the way (although in my opinion not quite complete &#8211; but this is a topic for another post) can be found e.g. <a href=\"https:\/\/jlelse.blog\/dev\/migrate-postgres-docker\/\">here<\/a>. Thus, explicit specification of the major version will result in upgrades, but only within version 11, what according to PostgreSQL documentation ensures backward compatibility and does not require migration of data files. Otherwise, once after execution docker pull you might be surprised that the system does not start because the new major version of PostgreSQL was released.<\/p>\n\n\n\n<p>2. HTTPS<\/p>\n\n\n\n<p>To implement secure connection we have used containerization as well.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"975\" height=\"1024\" src=\"https:\/\/blog.nubisoft.pl\/wp-content\/uploads\/2019\/11\/nc-ssl-975x1024.png\" alt=\"\" class=\"wp-image-80\" srcset=\"https:\/\/nubisoft.io\/blog\/wp-content\/uploads\/2019\/11\/nc-ssl-975x1024.png 975w, https:\/\/nubisoft.io\/blog\/wp-content\/uploads\/2019\/11\/nc-ssl-286x300.png 286w, https:\/\/nubisoft.io\/blog\/wp-content\/uploads\/2019\/11\/nc-ssl-768x807.png 768w, https:\/\/nubisoft.io\/blog\/wp-content\/uploads\/2019\/11\/nc-ssl-1462x1536.png 1462w, https:\/\/nubisoft.io\/blog\/wp-content\/uploads\/2019\/11\/nc-ssl-1200x1261.png 1200w, https:\/\/nubisoft.io\/blog\/wp-content\/uploads\/2019\/11\/nc-ssl.png 1820w\" sizes=\"auto, (max-width: 975px) 100vw, 975px\" \/><\/figure>\n\n\n\n<p>As you can see above, we decided to apply <a href=\"https:\/\/letsencrypt.org\/\">Let&#8217;s Encrypt<\/a> solution because it is good and because it is cheap \ud83d\ude09 We used three sidecar containers that form the truly self-playing orchestra with the first violins as <code>jrcs\/letsencrypt-nginx-proxy-companion<\/code>! All you need is to specify VIRTUAL_HOST, LETSENCRYPT_HOST, and LETSENCRYPT_EMAIL. With this solution, the certificate periodic renewal is being done totally automagically.<\/p>\n\n\n\n<p> So everything above will allow you to run NextCloud services in an efficient and secure manner. In the next part, we will show you how to configure NextCloud to use the resources of Simple Storage Services (a.k.a. S3). <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">To Be Continued&#8230;<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Update: We posted the second part already &#8211; feel invited <a href=\"https:\/\/nubisoft.io\/blog\/nextcloud-with-wasabi-s3-part2\/\">here<\/a>.<\/h3>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"block-76fa436b-d389-44d4-bc79-1c28db1538e7\">Update: We described how to upgrade NextCloud deployment to the newest version to keep your infrastructure secure and up-to-date  &#8211; feel invited <a rel=\"noreferrer noopener\" href=\"https:\/\/nubisoft.io\/blog\/how-to-upgrade-dockerized-nextcloud\/\" target=\"_blank\" data-type=\"URL\" data-id=\"https:\/\/nubisoft.io\/blog\/how-to-upgrade-dockerized-nextcloud\/\">here<\/a>.<\/h3>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In this post, we present how to deploy and configure NextCloud &#8211; the data-sharing platform &#8211; to store and secure files in an efficient way in terms of security and cost optimization. Nowadays, data-sharing among team members is a must. While in the case of source code there is practically one single way &#8211; Git [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":71,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_case_study_excerpt":"","footnotes":""},"categories":[5,4],"tags":[24,23,12,13,14,15],"class_list":["post-69","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-infrastructure","category-productivity","tag-docker","tag-dropbox","tag-file-sharing","tag-nextcloud","tag-s3","tag-wasabi"],"_links":{"self":[{"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/posts\/69","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/comments?post=69"}],"version-history":[{"count":10,"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/posts\/69\/revisions"}],"predecessor-version":[{"id":637,"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/posts\/69\/revisions\/637"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/media\/71"}],"wp:attachment":[{"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/media?parent=69"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/categories?post=69"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nubisoft.io\/blog\/wp-json\/wp\/v2\/tags?post=69"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}