Most NodeJS apps consist of hundreds of .js files which are included wherever needed using require. While this makes the dependencies of modules really obvious, most of the paths look like this in the end:
const createUuid = require('../../../support/utils/create_uuid'); The problem is that by default NodeJS uses relative paths for local modules (not stuff that’s inside the node_modules folder). Not only does this look really strange, it’s also hard to read and if you ever want to move files around you have to fix all paths in your application to make it work again (and since this is lazily evaluated, you’ll probably miss some for code paths that are not tested).
I’m running a Docker host on a Hetzner Cloud server. Since my GitLab instance grew quite big, I bought some additional storage that’s mounted to the machine as a additional device. All my Docker stuff uses local volumes located in /var/lib/docker/volumes. The question now was: How to move those volumes to the new storage? Docker volumes are basically just subdirectories in a special folder. For some applications it’s important to copy over all the extended attributes and other metadata like permissions, ownership and change dates.
In this series of screencasts I will create a Fever-compatible API layer for Nextcloud News, since I want to use this as my primary news reader. This API will be used to connect Nextcloud News to my Newsreader Apps on macOS and iOS. The API itself will be a separate application written in NodeJS that just uses the Nextcloud News database to query and write data. Later on I will probably create a Nextcloud App, but for now this series focuses on reverse engineering, functional programming and test-driven development in NodeJS.
Notes, besides photos, are the pieces of data I care most about. I want to review them even after years, have them available on the go and make sure I never lose them. Up until now, I stored my Notes in iCloud alongside Apples Notes app. This article is part of the byeCloud series in which I try to replace iCloud with self-hosted services.
The Apple Approach To Notes Initially, I wanted to retain this app and thought of going with IMAP Notes, which is what Apple does: They basically devote one IMAP folder to a list of mails with special headers.
In this little series I’d like to show you how to set up a blog, or in fact any website, using Hugo, a static site generator written in Go. I’ve used many static site generators from Jekyll to Octopress but in the end I got stuck with Hugo. As a evidence for how awesome Hugo is you can have a look at this very site (just kidding), which is also built using Hugo.
For some reasons I needed a Linux installation on my NAS. byhve is a lightweight virtualization solution for FreeBSD that makes that easy and efficient. However, the CLI of bhyve is somewhat bulky and bare making it hard to use, especially for the first time. This is what vm-bhyve solves - it provides a simple CLI for working with virtual machines.
The only requirement seems to be VT-x CPU support or whatever it may be called on AMD CPUs and ZFS as a file system.
While building my new NAS, I came across the question how to install a fileserver based on Samba on FreeBSD.
I’m working on an application where I use Docker and docker-compose to orchestrate a NodeJS container running my backend and a MongoDB container for my database. Using Docker for Mac, I’ve seen drastic slowdowns when initially establishing the database connection. When I used a Linux host, everything was extremely smooth. Since Docker for Mac creates a xhyve-based virtual machine running a Linux with the Docker daemon inside, I initially thought that this is just the usual slow in-VM performance compared to running Docker natively on the host system.
I recently started to convert all of my movies from MKV to MP4. The main reason for this is that I want to be able to play back videos from all my Apple devices without the need for additional software, like VLC or IINA.
For many people not that deep into video file formats I want to explain what the supported formats for video on macOS are and how to get there from almost any source material.
I’m currently working on a project in NodeJS. It’s an REST API server for a web application. Since it supports multi-tenancy but all tenants are created manually, I needed a quick and easy way to be able to call the createTenant method whenever I need it. I could have written a command-line tool that would do it for me, but actually I needed something like rails console for Ruby on Rails apps: A shell where I can just execute code in the context of my app.
In this article I will show you how to set up a Firefox Sync Server as a Docker container. In my case this will replace iCloud Bookmark / Tab synchronization. This article is part of the byeCloud series in which I try to replace iCloud with self-hosted services.
I’ve evaluated different solutions to synchronize tabs and bookmarks, but none of them seemed to satisfy my needs. Firefox Sync almost does.
iCloud Keychain has always been a love-hate relationship for me. I loved the simplicity of just being able to generate passwords right from the registration form of every page while it would also autofill passwords when I try to log in again. But I hated not being in charge of telling this thing to sync. Sometimes when I got a new device, the Keychain would just not start synching. Or passwords that I just created wouldn’t show up.
For some months now I’m running a private GitLab server. I really enjoy using it, especially with all the great features like the Docker Container Registry and GitLab Pages to host static pages, even with own domains. Normally I would prefer a more lightweight solution, such as Gitea but GitLab has so many advantages that, at least for me, this is currently the only way to go. However, it felt tedious setting it up, even with Docker.
For a project I was working on, I had a folder structure full of images and wanted to compress them to a specific maximum resolution in-place while retaining the aspect ratio using ImageMagick. The crucial point, as so often, were spaces and special characters in the folder names, making it really hard to use bash loops etc. to make this work. With the NULL character trick, luckily this works pretty well.
In the last days I tinkered a bit with things in Git that I haven’t tried yet. One was signed commits. Signed commits help other people to know that it’s actually you who committed changes. So when people trust you as a person, they can also trust your code because they can verify that it’s been actually done by you.
It’s pretty easy to set up and once configured, everything else will just happen automatically.
I maintain an archive of videos, especially documentaries from public media libraries from tv channels etc. on my NAS. Whilst I can use youtube-dl for the most part, it’s rather difficult for sites that use HTTP live streaming (HLS) to stream their content. In the most cases this is even AES-128 encrypted, which makes it difficult to download it. But since there’s players that support playback, it’s obviously possible to decode the stream on the client side.
Even though there are like one million guides on how to secure macOS properly, I decided to throw in my 5c on this topic. In contrast to many other guides, I’d just like to provide you with the best practices that I also use. This one is about the essential things that I always do when getting a new Mac to provide better security while preserving almost all macOS features like iCloud document saving or Time Machine usage.
During a setup change I had to temporarily move my incoming mailserver to Gmail and while I am now back in my own mail server, I again had this major headache of configuring the my DNS (using BIND) because I always forget the dots in the end of CNAME and MX records because usually I dont create nameserver records for external domains… So anyway here is the final configuration I used in Hetzners Domain Robot:
In this article I will show you how to set up your own RSS feed aggregator with sync support for many third-party clients through the Fever API as a replacement for Apple News/Google News/Feedly. This article is part of the byeCloud series in which I try to replace iCloud with self-hosted services.
Choosing the right software I fiddled around for a while with the News app for Nextcloud, ttr-rss and a few other alternatives, but ended up with Miniflux (now in version 2) because it is simple and lightweight and provides the most important thing for me, a Fever-compatible API without any plugins.
As it turns out, the Apple Time Capsule only uses SMB protocol version 1 and NTLM for authentication. While this works with macOS without any changes, it often creates problem when trying to mount the the Time Capsule’s SMB share on a Linux system. With this parameters, mounting should work out of the box, given you have the cifs-utils (Debian/Ubuntu) installed:
sudo mount.cifs //10.0.0.1/Data /mnt/timecapsule/ -o password='yourpw',sec=ntlm,uid=<local user>,vers=1.0 If you have any problems or further insights, feel free to leave a comment.
In this screencast we’re exploring the way Git saves files, directories commits and tags in it’s internal data structure, how we can inspect them and how they related to each other.
This video is meant to provide additional information for people that already use Git and want to know how it works under the hood.
Actually I like this approach to learn things and I think I’ll do another screencast on this topic, maybe providing additional information about how pack files and garbage collection works, how branches are stored and how integrity of data is ensured by their filenames.
Current macOS versions are weird… Often when I connect my MacBook Pro to the two screens on my desk, the wallpaper on one of that guys is either gone or changed to a different one.
After many attempts to search them in my bunch of 2k wallpapers, I eventually gave up and wrote a script that sets the wallpaper on all my screens like this:
osascript -e 'tell application "System Events" to set picture of every desktop to ("/some/path/wallpaper.
UPDATE Apr, 17 2018: Update imapsync to run as standalone docker container to make deployment easier.
In this article I will show you how to set up a fully-featured mail server including webmail as a Docker container. This article is part of the byeCloud series in which I try to replace iCloud with self-hosted services.
Let’s be honest: Setting up a mailserver really is a pain in the ass. Always.
In this article I will show you how I set up my photo synchronization using Nextcloud. This article is part of the byeCloud series in which I try to replace iCloud with self-hosted services.
One of the most important things in my setup is a solid and reliable way to synchronize Photos and videos I take. In one of the previous parts of this series I set up my Nextcloud instance.
In this article I will describe how I set up my Nextcloud instance aiming to replace iCloud Drive. This article is part of the byeCloud series in which I try to replace iCloud with self-hosted services.
The goals for a file syncing infrastructure are simple
I want a reliable solution that syncs files as-is, that does not corrupt them and does not cancel uploads all the time. Additionally, I want to be able to access my files on the go using a mobile app, as well as having files on my local hard disk to also be able to use it offline, just in case I have no network connection.
Introduction Some years ago I already played with ownCloud, trying to set up my personal cloud and get rid of third party services for keeping my stuff in sync across multiple devices. And while I already liked it at that time, there still were things I couldn’t do with it, so I eventually gave up on it.
Some months ago I decided to give it another shot and installed ownCloud (which is now migrated to Nextcloud) as well as some other services aiming to replace iCloud, the cloud service by Apple that I used until then.
Just as a short update: Starting from the newest AirPods update, they also finally work as a headset on the Mac. Before there was a issue that caused audio quality to drop when using the AirPods both as headphones and microphones at the same time.
macOS has the osascript command line tool that allows you to run embedded AppleScript right from within shell scripts. As AppleScript also has capabilities to show notifications, you can utilize this to show messages in the macOS Notification Center.
Here is an example:
osascript -e 'display notification "Something happened" with title "Test"' If you have any further tips that will help people supercharge their shell scripts on macOS, feel free to leave them as a comment.
I have so many accounts for various web-based solutions, I barely can remember a few. And they send emails. So many that it’s sometimes hard to not loose the overview in my mailbox.
When you run your own mail server, it’s easy to set up a dynamic aliases for your mailserver based on a regular expression pattern that allows to e.g. have a own email address for each service you register.
I wanted to expose a single jail of my FreeBSD NAS to a network of a client via OpenVPN while it’s reachable both from my network and from the clients’ network. It should send all of its traffic through that VPN tunnel so that it appears like it is just another computer on that foreign network.
Luckily FreeBSD offers a great way to solve this by creating a separate routing table apart from my main routing table that is used when starting OpenVPN (so that it can populate it’s routes there) and when starting the jail (the jail in fact will consider that routing table as the only routing table available and therefore use it for anything).
After finishing with the hardware and software parts of my new NAS, I decided to append another little project which is aimed to provide a simplified control panel for macOS in the menu bar on the upper right of the screen.
Objective What I wanted to achieve is a possibility to mount my various shares with one click as well as having controls for power on/off and SSH. Additionally the control should indicate whether the NAS is currently powered on or not.
UPDATE 2018-06-02: I’ve created a new version of this article called “Securing macOS” available here: https://www.davd.io/securing-macos/
Setting up a NodeJS application on a FreeBSD 10 system was impossible when using ZFS as a file system. This was a real pain for me because when I tried out various stuff in jails rendered the whole system unusable, forcing me to reboot and this is not something one would do in a production environment.
Here’s the bug report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=209158
Although I couldn’t read it from the bug it seems to be resolved now with FreeBSD 11-RELEASE because I’m running two Node applications on FreeBSD without any of the issues that were reproducible each time.
If you encounter any problems, please check the up-to-date documentation at https://github.com/dprandzioch/docker-ddns .
In some projects I need access to various hosts with a dynamic IP from time to time. Dyndns services offer a great solution by providing a DNS with records that are updated by the clients once their IP addresses change.
There are various existing services out there that are either free or paid, but if you want to self-host a dyndns service, you have to set up a DNS by your own as well as some endpoint that your clients can connect to in order to update their records.
Sometimes it seems that Apples iCloud sync seems to do not exactly the right thing. During initial upload of my photos I multiple times had the issue, that it was not possible at all to upload the photos.
When looking at the sync pane of the Photos app, it was something like “18 hours remaining…” but nothing happened, even after two days. The system process responsible for managing the photo upload is called cloudphotosd.
The company I’m working at provides eCommerce solutions for many years now. A few years ago we
decided to give up on our own product and started to become an agency that would work with a
existing eCommerce application from now on. In our own software, we provided a SOAP API which
hadn’t changed for years that had some client-side implementations in various ERP systems and
when we switched over, we decided to provide a compatibility plugin for the new software that
would expose the SOAP facade we built years ago and translate all requests to the REST API (which
we called internally without going over HTTP again).
In Part 1 of this series I already explained my goals building a new NAS.
In this post I show how I assembled the hardware in order to ensure reliance and redundancy.
FreeBSD is the ideal system to use when building a server. It’s reliable and rock-solid and it’s file system ZFS not
only offers anything you would expect from a file system but is also easy to set up and to maintain. This is why I chose
it to power my NAS. In Part 1 and Part 2
of this series I already described my intentions and the hardware assembly. Now it’s time to bring it to life.
In Part 3 of this series I described how to install FreeBSD and set it up properly. Now that the base system setup is
complete, we can start providing services…
While building my new NAS, I came across the question how to provide a Time Machine backup solution for my OS X clients.
As I run OS X on all my machines I want to back up all data to my NAS.
netatalk allows to create file shares for OS X
to provide a simple solution for system backups.
forked-daapd allows you to set up an iTunes Media server that hosts all music, podcasts and audiobooks and shows
up in iTunes like a shared library. While other
daapd implementations don’t work anymore with the current iTunes
After a long while I finally decided to build a new NAS / home server for my various needs. Though there are many
solutions available, I chose to build one on my own as I want as much flexibility as possible. So I set out to buy
all components needed for the system with upgradability and budget in mind.
I’m changing my hardware quite frequently as I often end up unsatisfied with my current setup. Setting up a computer
from scratch is a pain in the a** but restoring a backup implies carrying around configuration files, useless software
and other stuff for years. So I’ve decided to create a script that would set up a new computer from scratch and
configure it the way I want it to be.
FreeBSD is shipped with sendmail as the default MTA, which is configured to local delivery on a vanilla installation.
Therefore many people don’t even recognize one of FreeBSDs great features for system administrators: FreeBSD sends
system status emails through periodic(8)…
Beside the pre-configured profiles, OS X’s sandbox wrapper command
rsync as a very important tool in a system administrators’ toolbox. It allows to synchronize files and directories and is preferred over cp by many, especially when operating on a large file base because it allows to resume copying in case it is canceled. But rsync can do so much more… For example syncing files and folders over SSH, like scp copies files and folders over SSH but again, with some advantages.
Jails in FreeBSD provide a simple yet flexible way to set up a proper server layout. In the most setups the actual server only acts as the host system for the jails while the applications themselves run within those independent containers. Traditionally every jail has it’s own IP for the user to be able to address the individual services. But if you’re still using IPv4 this might get you in trouble as the most hosters don’t offer more than one single public IP address per server.
In my daily work one of my jobs is to assure code quality of our web applications written in PHP. Beside the usual checks like manual code review, training (e.g. using XP programming sessions) and automated unit tests using PHPUnit, there are some metrics that can be measured automatically making my work easier and helping to reduce error rates in the review process. In the following article I want to introduce the most common methods and tools addressing static code analysis in PHP.
sandbox-exec provides a flexible configuration
syntax that allows one to create a customized sandbox that either blacklists or whitelists specific abilities of the
application executed within.
I have used Caddy for a while now on FreeBSD. But though it lacks a working init script, I decided to write one on my own.
Here’s the result, description below:
There are some tools that make a developers’ daily work much easier. Here’s my top 20.
When using Git extensively, you’ll likely have multiple branches which need to
be cleaned up from time to time. In some repositories I have hundreds of
release/ branches, which are already merged to the
master branch and deleting them manually would be a mess.
So here’s how to delete all branches from the server, that are already merged
master branch. In this case, branches need to start with either
release/ but you can simply adjust this to match your needs.
It’s more than likely that your email provider of choice, especially the ones
that offer mail services free of charge, will not support receiving email to
custom domain names like, in my case, davd.net. Running your own mail server would
solve this problem but running a fully featured mail stack including POP, IMAP,
Sieve filters et cetera requires a fairly powerful machine.
Additionally, if not configured properly, there’s big potential for abuse,
As an alternative, it’s possible to just run a MTA which redirects all incoming email
to an external mail server.
This can be ran on almost any machine, even on a low-budget computer like the
Raspberry Pi or a cheap virtual server.
During the last few months I managed to automate many recurring tasks on my NAS.
One good example for those task is updating my podcast archive. I tried to accomplish
this using a lightweight shell script which, running as a cronjob, would hold my
podcast archive up to date and notify me about new episodes via push notifications.
Often the question arises why one should prefer FreeBSD over any arbitrary GNU/Linux distribution, that is more widely spread and additionally offers commercial support. Though I have mostly positive experiences using Linux, there are many reasons why I would always choose FreeBSD for setting up a server.
In the following quick start guide I will show you the basic usage of Puppet.
If you want to follow me along, you’ll only need the text editor of your choice
(I’m using Sublime Text here) and a command line. Also you should be using a
supported operating system, I’ll describe the installation exemplary on Fedora
Linux and Mac OS X Yosemite.
It’s already been while since Apple removed the “Java Preference Pane” from OS X,
with which it was possible to change between various installed Java versions.
I shortly needed to compile some old Java application against the JDK version 1.6
and I was questioning how to change the compiler version.
In this guide I’ll show you how to install the package manager Bower on Mac OS X.
Bower is a NodeJS application, so you’ll need to install the Node Package Manager
npm is available as a package in the Homebrew repositories.
If you’re running a network with a .local domain, you might experience
issues running OS X Yosemite (10.10). Heres how to resolve this issue.
In this guide I’ll show you, how to prevent permission changes
within a Git repository to be recognized as a file change.
At least after changing file permissions using the
your version controlled project, the output of
git status will be a mess making
it nearly impossible to identify changes within your project’s source code.
So it’s already been a month ago when I got my new 13-inch MacBook Pro w/ Retina
Display. So everyone who knows me could already promise what would happen next:
I’d install Linux on it. For this one I once again chose to use my favorite Linux
distribution Fedora, which is currently available in version 20. Meanwhile I tried
to install Ubuntu but this lead to problems with ACPI, the disk controller and
last but not least the hi-res display featuring a pixel density from 227 ppi.
There are some things to have an eye on during the installation process but
generally most things are working now so that the system is more or less ready
for production usage.
Yesterday I recognized that the latter USB port doesn’t seems to be
broken. I tried to sync my iPad, but it got no connection the the MacBook.
dmesg log showed no activity after connecting any device to the
USB port. The front USB port worked without any issues. What first seemed to be
an hardware issues turned out to be a interesting foible of the MacBook family.
To cut a long story short, the port isn’t broken. I found the solution somewhere
on the internet but don’t know exactly where by now, so thanks to the original
author on this way.
There’s at least one case where I can’t confirm Apples motto “It just works!”.
Since I’m using my AppleTV (2nd generation), I’m permanently experiencing problems
while transmitting data between my Mac running iTunes Home Sharing and the AppleTV
itself. Most of the time the Home Sharing resources are simply not visible in
the AppleTV’s’ user interface. According to various Apple support items it seems I’m
not the only one experiencing those problems, so here’s my solution…
This one seemed easier then it was in the end ;-) Here’s how it works…