blog

How to update your downloaded components?

This blogpost explains the reasoning of how we take on versioning at Compony. If you just trust us and don't need an explanation, go on and skip to the bottom few paragraphs. :)

 

How do others do it? #

The question of: "Can I update a component" is not a simple one.

Sometimes it means we need an extra toolset to update parts of our code such as NPM or Composer. Usually it also means that we can't touch the contents of what we downloaded. Next to that, it's usually the case that we download more than we need. And often it also means that we are scared of updates. 

Whatever solution we chose to approach this problem, will have a long lasting impact on our projects and on the way we work.

So let's have a look around us and see how others are approaching this problem of updatable parts of our code. 

 

The Drupal community and Composer #

There are many benefits to using Composer. In short, it allows us to systematically manage a sprawling list of dependencies (and their subsidiary dependencies). It assists with locating, downloading, validating, and loading said packages, all while ensuring that exactly the right versions for each package are used. (ref: Drupal.org)

Drupal relies on Composer for dependencies, Drupal Core, and Contrib modules. That's why you now have to write a Composer command in order to be able to install a contributed module.

Composer is a perfect match for Drupal. it downloads Drupal core, the contrib modules and all of it's dependencies. But it also comes with a very strong side-warning: don't edit anything you download through Composer.

Think about it: you wouldn't edit files in the Drupal Core folder, would you? In the same way you would also not go and change something in the source code of a contributed module.

So that means that whatever we download through a tool as Composer, we shouldn't edit. Not only should we not edit it, we should also not commit it to Git! If you think about it a bit more: the only thing that you would commit is the composer.lock file which is just a configuration file so Composer knows what it did previously. 

We are however still able to change the behaviour of Drupal, and it's modules by using hooks and configuration. Hooks are the way how newly written code can interact or change the already written code. Some already written code will be stored in the by-composer-downloaded-untouchable files, so hooks allow us a way in.

Hooks work really well because they are scoped off in terms of what you can do in each hook. If we think about updatable code, this is a stunningly amazing concept. Each hook is limited to the things it is supposed to change. Drupal knows how to make all those hooks nicely work together, and therefor you could see Drupal-core as a bundler of Drupal-code.

So let's summarise this conceptually: the tool that Drupal relies on for downloadable components (called modules in Drupal-land) makes them untouchable and makes them live outside of version-control. Each part that is downloaded is also very nicely scoped off by design.

 

The Javascript community and NPM #

NPM is a package manager. that as a tool automates the process of installing, upgrading, configuring, and removing javascript packages.

As a developer, we can download from a big range of Javascript solutions made by others and not worry about dependencies.

NPM has a lot of similarities with Composer: anything you download with NPM you are not supposed to edit the source-code or commit to Git.

Just as with hooks in PHP, you can change the behaviour of the downloaded Javascript-packages by passing along configuration-options, These Javascript packages are usually divided in to abstractions so small that you need a tool to compile all of these abstractions in to 1 file again. 

These abstractions are so small so that the amount of combinations you can make is huge. They are so huge, that the file-size of downloaded packages is usually quite huge too. Because you are not using every feature of every tiny package that you've downloaded, the community invented tree-shaking, where you get rid of code that your project isn't using, to make the file-size smaller again.

Even with tree-shaking, no developer ever goes in to their node_modules folder to see what packages exactly they have. But to be fair, we shouldn't feel too guilty about this. We have a ton of stranger's code in our setup, but the good thing is: it stays in our setup mostly. We don't fetch the entire node_modules folder on each page load of the website we build.

Every now and then a project comes along that lasts long enough that you should update your npm-packages for. I have never succeeded in updating my npm-packages with ease, it has every single time been an absolute nightmare. I think the main issue here is that you are trying to update a bunch of abstractions (read packages) that don't really care about each other,

Because the packages are soo high in number in our project, it's impossible to read all of the release notes of each package. So there are only 2 reasons why we would ever take the time to update anything:

A) We found out we have vulnerabilities in our code.
So we use a update script to automatically update all of our packages, and we hope everything still works exactly like before. 

B) We want a new version of 1 package.
So we update only this one package, and we hope that we don't need to run the update script like we would do in scenario A, as that involves hoping nothing will break. 

In each scenario there is a lot of hope involved that things won't crash and burn. Which usually means that we either avoid updating, or that we simply remove all of our previously installed packages and just run the install script again.

The people over at NPM are doing a stellar job at creating these tools. But for me, I always imagine that If Henry Ford would be a CEO of NPM, he would have been laughing with the request of his customers that demanded updatable JavaScript packages. Instead, he would have shifted the companies focus towards maintainable and secure packages.

If we need to remove everything in order to consistently be able to "update", can we actually still call it "updating" and not "burn it down, and build it up"?

So let's summarise this conceptually: the tool that the Javascript community relies on for downloadable components (called packages in Node-land) makes them untouchable for us as a developer and makes them live outside of version-control. We have the capability of updating these packages, but often it is not worth the pain to actually do so.

 

The problem #

Having untouchable downloaded updatable files in PHP and JS context makes a lot of sense. 

But it becomes another story with Frontend components consisting of HTML, CSS and JavaScript. 

The concept of updatable HTML is quite a neverending-loop to explain: Making HTML-components updatable, would imply that you can pass along configuration to customise that HTML. If we would do this, then the next question will be: what part of the HTML code should be configurable? It would be not ridiculous to assume that you would want every part of your HTML-component to be customisable (from tag, to attributes, to its contents). So then we would abstract an updatable HTML-component, where every single part being changable. But if everything of the original component can be changed then there is nothing left to update as it becomes a meaningless abstraction.

On top of that: CSS isn't scoped by design. This means that anything you write in CSS can break any other component. So creating a registry of unscoped updatable components, would involve a lot of conventions in order to not have it end badly. But even if we do invent all of these conventions and we make this work, then we end up with the same neverending-loop we have in terms of flexible vs updatable. But let's assume we could fix that loop, then it would also mean that we lock every component in terms of evolving in any meaningful way.

This is also the reason why Drupal-modules are stuck in updating their HTML/CSS-output. They can't change a class, because they don't know how people already relied on them in terms of theming, or writing JS. In the same way they can't change anything semantically because of people having built on top of this already. HTML nor CSS don't work in a way that you can hook in to them such as in PHP-compiler or JS-transpilation. If you want a <button>-tag to become an <a>-tag. You need to change the actual tag before it's send to the browser. PHP gets compiled on the server, JS gets compiled by our toolset but the HTML and CSS gets interpretted by the browser, and that's final.  

 

Our approach #

If we ignore the request for updatable components due to the impossibility explained above. Then we have a whole new set of opportunities.

 

The Sketchbook-principle
If you download components, we encourage you to take the sketchbook-principle. Drag the component in to your theme, load the page of your project where you can see the output of the downloaded component, and start from there.

Likely you will want to change some colors to match your project straight into the Scss-file. You might want to change some classes straight in the .html.twig file. And that's it. you don't have an extra abstraction of things you can't touch. In every file you've downloaded you can change code however the need of your project is. Likely 90% of your component will already be good to go, and the remaining 10% is some layout, color, and little tweaks that make the theming unique.

 

Dependency-free
Components usually don't have dependencies, as each component should ideally stand on it's own. However sometimes there are some dependencies between components. Those components always are part of the same collection on Compony.io.

Due to the flexible nature of the contents of each component, listing them as dependencies wouldn't be correct either. As they can easily evolve to gain or loose dependencies with each line of code that you change. 

 

Commit-versioning
When you download a component from Compony.io, the component will contain a automatically generated hidden .component.yml file. This file lists all of the information at the time that you downloaded this component. The contents looks similar to this:

name: 'Form with basic CSS'
folder_name: form
contributors:
  - Compony
more_information: 'https://www.compony.io//component/form-with-css'
repository:
  gitlab_project_id: '7359579'
  gitlab_project_url: 'https://gitlab.com/componies/flat-design/Core/form'
  downloaded_version: 'https://gitlab.com/componies/flat-design/Core/form/commit/f92dd07c3d2832df6ecfd07e334ddfb147020f34'
  downloaded_version_sha: f92dd07c3d2832df6ecfd07e334ddfb147020f34
  downloaded_version_sha_short: f92dd07c
  diff_downloaded_and_latest_version: 'https://gitlab.com/componies/flat-design/Core/form/compare/f92dd07c3d2832df6ecfd07e334ddfb147020f34...master'
  git_clone_specific_version_command: 'git clone https://gitlab.com/componies/flat-design/Core/form.git form && cd form && git checkout f92dd07c3d2832df6ecfd07e334ddfb147020f34'
downloaded_on: 07-26-2019
supported_by: no-one
built_upon: 'Drupal Core'

So if in a years time, you discover a bug in your form-component, you could check the diff_downloaded_and_latest_version url in this file. And see what changed in the component's repository since you've last downloaded it, to give you some hint to what your issue could be. (Example: diff with last commit)

Depending on how much you changed inside the downloaded component, you could decide to update your component completely, or only partially. (Docs: .component.yml)

 

Flexible is better than updatable
Writing good Frontend code on an existing project, usually means a lot of refactoring, nearly all the time. If we don't keep that flexibility of completely changing code inside our components, then Compony is going to be more in our way than helping us as frontend developers. The flexibility also means you can update your components on the platform without worrying about backwards compatibility.

The flexibility also allows us to have updatable sketchbook. ( as in: you can just click download on a collection detail page). You will get all the latest versions of all the components that it contains, so your sketchbook is completely up to date.