Skip to content

Instantly share code, notes, and snippets.

@chadluo
Last active May 7, 2024 15:07
Show Gist options
  • Save chadluo/16bcb6740049cda97a40c196196b8e90 to your computer and use it in GitHub Desktop.
Save chadluo/16bcb6740049cda97a40c196196b8e90 to your computer and use it in GitHub Desktop.
An oversimplified idea behind web development and tooling etc., like how did it come here

I remembered reading an article like this years ago but couldn't find it any more. I'd rather rewrite one myself.

Browsers

The term "Web" in most contexts refer to stuff happening in the browser or on the browser tech stack, for example when comparing different implementations of an app (platform native, containerized/VM or in a browser). Web apps can reach cross-platform easier than other tech stacks as most devices will have a browser, but they fall behind in performance and system API availability.

Browsers send requests to servers and receive text based HTML code represented various elements -- text, blocks, links to external resources like images & videos -- to render onto the viewport.

HTML supports semantic elements (bullet point lists, tables etc.) and basic styling (for example fgColor and bgColor), but people want better control and performance, so CSS was developed.

Early HTML was static, meaning that the content won't change and there's limited interactions. Netscape decided to support some programming mechanism and hired Brendan Eich to design and implement a language, which is later referred to as JavaScript.

It's worth mentioning that the JavaScript language and runtime are separate ideas. Some features are provided by JavaScript itself, like passing functions as parameters to other functions, and others are provided by the runtime -- at this stage, browsers -- like DOM. The differences becomes vital when comparing different runtimes.

JavaScript made it possible to do simple tasks on a web page like form validation, but it was until the release of Gmail that people realized the capabilities and potential of JavaScript and full-fledged web apps were possible. For example, with AJAX where JavaScript sends requests to servers and receives responses, you can perform complicated interactions and the page could update itself so you don't have to refresh.

A simple HTML document with CSS and JavaScript could look like this:

<!DOCTYPE html>
<html>
  <head>
    <title>Sample Page</title>
    <style>
      /* This part is CSS */
      h1 {
        font-size: 40px;
      }
    </style>
  </head>
  <body>
    <h1>Hello</h1>
    <script>
      /* This part is JavaScript */
      console.log("Hello");
    </script>
  </body>
</html>

Some notes:

  • An HTML document is made of nested tags (wrapped in angle brackets). Tag names can be from DOM or custom-fined, as long as they follow the rules with each tag. Some tags needs to be properly closed but not all. Tags can contain text or other tag structure.
  • CSS and JavaScript are added to the HTML under style and script tags respectively, and their content appear as HTML text. If a browser supports these languages, it will interpret the content and won't show them on the viewport.
  • CSS and JavaScript don't have to be in the HTML document; they can be in independent files referenced from HTML and the browser will try to get these files and interpret them.

Although you can hand write everything since all HTML, CSS and JavaScript is text based, earlier browsers differ quite a lot regarding implementations. various tools are created to generate the code and make it easier to maintain.

Dependencies

3rd party code can be categorized into libraries and frameworks. One way to tell the difference is that, when you structure the code your way and need some functionality, you can use a library to accomplish it locally; when you follow the structure requested by the 3rd party code and provide your components for the 3rd party code to use, then that is a framework. Lodash, Underscore and jQuery are good examples of libraries, while Next.js, Nest and Nuxt are good examples of frameworks.

When you want to use 3rd party code you can download a copy, save it together with your code and reference in your code, or use a hosted version, for example from docs of jQuery and Bootstrap. Typically for CSS you add it to <head> and JavaScript to the end of <body> before the closing tag.

Node.js

Ryan Dahl wanted to create a non-blocking server to solve some performance issues, and noticing that

  • JavaScript was friendly to the paradigm at language level, and
  • The JavaScript runtime V8 from Google Chrome was open sourced and pretty performant

he decided to use JavaScript for the project and released Node.js.

As another runtime of JavaScript, Node.js enabled using one language for multiple tasks, but the differences in the runtimes make it pretty different to write code for two sides. Node.js provided access to port binding and filesystem, which makes it ideal for servers and also desktop toolsets. But not in browser means it doesn't have access to functions like window.prompt() and window.alert().

Node.js distributions come with the built-in package & project manager NPM. For a project, you can use npm to install dependencies and use node to run both your code and code from dependencies. Since most of Web development tools reside in NPM ecosystem, NPM is the most important tool in web development.

Starting from an empty directory, install Node and run

npm init

Upon answering the prompted questions, npm generates a package.json file containing the project info looking like:

{
  "name": "sample-project",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC"
}

You can notice the generated scripts section. This specifies commands associated with your project, for example to build, test or wind up a dev environment. When you want to run some commands, for example test from above, you can use

npm run test

and you can add your own to the file.

Most 3rd party packages will provide you of how to install them. For example if you want to install TypeScript, you can find the instruction on the project page:

npm install -D typescript

After this you'll notice an added section in package.json:

{
  "name": "sample-project",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
-  "license": "ISC"
+  "license": "ISC",
+  "devDependencies": {
+    "typescript": "^5.4.3"
+  }
}

devDependencies means the added dependency is used during development and not necessarily part of your production code. For the code that will appear in your final product they will come under dependencies. Where the dependency sits is controlled with the -D option in the install command from the TypeScript example above. When you install a package they will usually provide the correct install command for you to copy and paste.

You will notice a new directory node_modules, which contains the dependency code. In the past the dependencies can be brought in under dependencies creating a huge tree of duplicated copies of packages and taking excessive space. This is fixed in recent versions of NPM and some other package managers.

You will also notice a new package-lock.json which specifies the exact version and hash code of dependencies. This file is supposed to be verbose and managed by automated tools, whereas package.json could require manual updates from time to time. Also note that package-lock.json is used by NPM and other package managers will use their own, like Yarn uses yarn.lock and pnpm uses pnpm-lock.yaml, but they use the same package.json.

When you share your code with others, normally you only share package.json and package-lock.json (or equivalent) alongside with other project code and leave out node_modules. When other people have obtained their copy, they can run npm install and their npm will download the specified version according to specified json files, same as when you obtain other people's code.

NPM can manage the version of dependencies you install, but when you run node or npm itself, it will use the version (globally) installed on your computer resolved from your $PATH. If you need to manage versions of node/npm you can use tools like nvm.

Bundlers

For simple projects it's straightforward to simply link the JavaScript file, but when you have installed multiple code dependencies -- packages under package.json/dependencies -- how are they included in the code? If you have multiple JS files, do you need to include each of them to the HTML file? What if file structure changes, either because you added new file or performed a major refactor?

Bundlers analyze the code and put them together, and can perform other tasks during the process, like compiling/transpiling different languages to browser compatible JavaScript. There has been different popular options and for now Webpack is probably the most important one.

Webpack can combine multiple resource files and generate one output JavaScript file to include in the HTML document. The resource files are not limited to JavaScript files: you can use loaders to process other files as well. In a simple case, you can install Webpack, declare the generated JavaScript file path and refer to that file from HTML. You can follow the guides at Getting Started.

Framework Starters

Although it's beneficial to understand how the tools work and even better to follow the steps to set up a project, it's not necessary. Modern frameworks usually come with starters to setup these required files. You can see examples for Next.js and Vue.

Backend

While some frameworks are full-stack (For example Next.js or Nest.js), others only worry about the frontend part and you will need some backend part to handle requests. Express has been a common option on Node.js stack but it's not as active. You are not limited to JavaScript either as long as your frontend and backend follows the same pattern.

Other tooling

As a fast growing community, all of the tools mentioned above have alternatives. For NPM there are Yarn and pnpm, for Webpack there are esbuild and Vite, even for the language JavaScript itself there are languages compiling into JavaScript (since it's still the only supported language in browsers), most notably TypeScript, and for Node there are Bun and Deno. You should choose your tools and dependencies prudently.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment