JS - Backend - CL1 Flashcards
Node.js fundamental concepts
At the European JSConf in 2009, Ryan Dahl, a young programmer, presented a project he had
been working on. This project was a platform that combined Google’s V8 JavaScript engine,
an event loop, and a low-level I/O API. This project was not like other server-side JavaScript
platforms where all the I/O primitives were event-driven and there was no way around it.
By leveraging the power and simplicity of JavaScript, this project turned the diffi cult task of
writing event-driven server-side applications into an easy one. The project received a standing
ovation and has since then been met with unprecedented growth, popularity, and adoption.
The project was named Node.js and is now known to developers simply as Node. Node
provides a purely event-driven, non-blocking infrastructure for building highly concurrent
software.
Ever since its introduction, Node has received attention from some of the biggest players in the
industry. They have used Node to deploy networked services that are fast and scalable. Node
is so attractive for several reasons.
One reason is JavaScript. JavaScript is the most widely used programming language on the planet.
Most web programmers are used to writing JavaScript in the browser, and the server is a natural
extension of that.
The other reason is Node’s simplicity. Node’s core functionalities are kept to a minimum and all the
existing APIs are quite elegant, exposing the minimum amount of complexity to the programmers.
When you want to build something more complex, you can easily pick, install, and use several of the
available third-party modules.
Another reason Node is attractive is because of how easy it is to get started using it. You can
download and install it very easily and then get it up and running in a matter of minutes.
Traditional programming does I/O the same way as it does local function calls: Processing
cannot continue until an operation fi nishes. This programming model of blocking when doing
I/O operations derives from the early days of time-sharing systems in which each process
corresponded to one human user. The purpose was to isolate users from one another. In those
systems, a user would typically need to fi nish one operation before deciding what the next
operation would be. But with widespread use of computer networks and the Internet, this
model of “one user, one process” did not scale well. Managing many processes places a big
burden on the operating system — in memory and context switching costs — and the
performance of these tasks starts to decay after a certain number is reached.
Multi-threading is one alternative to this programming model. A thread is a kind of light-
weight process that shares memory with every other thread within the same process. Threads
were created as an ad hoc extension of the previous model to accommodate several concurrent
threads of execution. When one thread is waiting for an I/O operation, another thread
can take over the CPU. When the I/O operation fi nishes, that thread can wake up, which
means the thread that was running can be interrupted and eventually be resumed later.
Furthermore, some systems allow threads to execute in parallel in different CPU cores.
This means that programmers do not know what set of threads is executing at any given time,
so they must be careful with concurrent access to the shared memory state. They have to use
synchronization primitives like locks and semaphores to synchronize access to some data structures, forcing them to foresee every possible way threads can be scheduled to execute to try to
prevent problems. If the application relies heavily on a shared state between threads, this type of
programming can easily lead to strange bugs that happen at random and are usually diffi cult to fi nd.
An alternative to having the operating system scheduling the thread execution for you is to use
cooperative multi-threading. In this scheme you are responsible for explicitly relinquishing the CPU
to give time for another thread to execute. Because you are now responsible for thread scheduling,
this can relax the synchronization requirements. However, this approach can become complex and
error-prone for the same reasons as regular multi-threading.
INTRODUCING THE EVENT-DRIVEN PROGRAMMING STYLE
Event-driven programming is a programming style whereby the fl ow of execution is determined by
events. Events are handled by event handlers or event callbacks. An event callback is a function that
is invoked when something signifi cant happens — such as when the result of a database query is
available or when the user clicks on a button.
HOW NODE AND JAVASCRIPT MAKE WRITING ASYNCHRONOUS
APPLICATIONS EASIER
Ryan Dahl, the author of Node, began his project building a C platform, but maintaining the con-
text between function calls was too complicated and led to complex code. He then turned to Lua,
but Lua already had several blocking I/O libraries. This mix of blocking and non-blocking could
confuse developers and prevent many of them from building scalable applications, thus Lua was not
ideal either.
Dahl then turned to JavaScript. JavaScript has closures and fi rst-class functions, which makes it a
powerful match for event-driven programming. The power of JavaScript is one of the main reasons
Node has become so popular.
Why Node.js?
If a picture speaks a thousand words, what would it take to speak a thousand pic-
tures? Or for that matter, an infinite number of pictures? My first introduction to
Node.js was through WordSquared, 1 seen in Figure 1.1. This is an online, real-time,
infinite game of Scrabble built using the same technologies that we’ll discuss in
this book. As soon as I set eyes on the game, I had to find out more about the tech-
nology behind it, and I hope you feel the same.
What’s incredible about the game is that it was prototyped in just 48 hours as part
of Node.js Knockout. 2 Bryan Cantrill, VP of Engineering at Joyent (which manufac-
tures Node.js) has said that when doing things in Node.js, you sometimes get the
feeling of “Is this it? Surely it needs to be more complicated.” This is a sentiment I share. Node.js is a joy to work with, and I intend to share that with you through
the code we’ll write throughout this book.
Node.js is a server-side JavaScript platform that consists of a deliberately minimalist
core library alongside a rich ecosystem. It runs atop the V8 JavaScript engine, which
makes it very fast thanks to the talented engineers at Google. JavaScript is popular
on the client side, but it was chosen as the target language primarily for engineering
considerations, the details of which will be discussed as the chapter unfolds.
The home page of Node.js describes it thus:
“Node.js is a platform built on Chrome’s JavaScript runtime for
easily building fast, scalable network applications. Node.js uses an
event-driven, non-blocking I/O model that makes it lightweight and
efficient, perfect for data-intensive real-time applications that run
across distributed devices.”
—http://nodejs.org/
This may seem cryptic to newcomers, but it succinctly summarizes some of the key
strengths of Node.js and is worth exploring in more detail. People are often taken
aback when they hear that JavaScript is the targeted programming language. That’s
because there’s a perception in the programming community that JavaScript is not
a “real” language such as C, C++, or Java. Yet JavaScript did have its genesis as an
interpreted language for the browser; the “Java” part of the name was actually chosen
to capitalize upon the perceived popularity of the Java programming language at
the time.
Since its humble beginnings, JavaScript has proliferated and is now supported in
every major web browser, including those on mobile devices. Not only is it a popular
language, but the tools and frameworks currently available for it make it qualify as
a powerful engineering tool. JavaScript as a server-side platform supports continuous
integration, continuous deployment, connections to relational databases, service-
oriented architecture, and just about every other technique available to its more
well-established brethren.
In conjunction with Google’s V8 JavaScript engine, it is now extremely fast; in fact,
it’s several times faster than other scripted languages such as Ruby and Python.
Against Python3, JavaScript V8 Engine has a median benchmark 13 times as fast
with a roughly similar code size. 3 Against Ruby 1.9, the median benchmark is eight
times as fast. 4 These are incredible benchmarks for a dynamic language, and are
due in no small part to V8 optimizations such as compilation into machine code
pre-execution.
The official description talks about the event-driven, non-blocking I/O model. Tra-
ditionally, programming is done in a synchronous manner: a line of code is executed,
the system waits for the result, the result is processed, and then execution resumes.
Sometimes waiting for the result can take a long time; for example, reading from a
database or writing to a network.
In languages such as Java and C#, one solution is to spawn a new thread. A thread
may be thought of as a lightweight process that performs tasks. Threaded program-
ming can be difficult because multiple threads can be trying to access the same re-
source concurrently. Without dissecting the intricacies of multi-threaded program-
ming, you can imagine it would be disastrous for one thread to be incrementing a
counter while another thread is decrementing the counter at the same time.
JavaScript approaches the problem differently. There is only ever a single thread.
When doing slow I/O operations such as reading a database, the program does not
wait. Instead, it immediately continues to the next line of code. When the I/O oper-
ation returns, it triggers a callback function and the result can be processed. If the
mechanics of this seems slightly counterintuitive, rest assured that by the end of
the book it will be second nature, because we’ll be seeing this pattern over and over
again. Node.js offers a simple, fast, event-driven programming model well-suited
to the asynchronous nature of modern-day web applications.
Strengths and Weaknesses
Node.js is not a panacea. There is a certain class of problems in which its strengths
shine through. Computer programs can be generally classified according to whether
they are CPU bound or I/O bound. CPU bound problems benefit from an increase
in the number of clock cycles available for computation. Prime number calculation
is a good example. Node.js, however, is not designed to deal with these CPU bound
problems. Even before Node.js formally existed, Ryan Dahl proposed the following: 6
“There is a strict requirement that any request calculates for at most,
say, 5ms before returning to the event loop. If you break the 5ms
limit, then your request handler is never run again (request handlers
will be tied to a certain path/method, I think). If your uploaded
server-side JavaScript is stupid and is blocking (… it calculates the
first 10,000 primes) then it will be killed before completion.
…
Web developers need this sort of environment where it is not pos-
sible for them to do stupid things. Ruby, Python, C++, [and] PHP
are all terrible languages for web development because they allow
too much freedom.”
—Ryan Dahl
I/O bound problems are alleviated by increased throughput in I/O such as disk,
memory, and network bandwidth, and improved data caching. Many problems are
I/O bound, and it’s in this domain that Node.js truly shines. Take, for example, the C10K problem, 7 which poses the dilemma of how to handle ten thousand or more
concurrent connections for a web server. Some technology platforms are ill-equipped
for managing this type of capacity and require various patches and workarounds.
Node.js excels at this task because it’s based on a nonblocking, asynchronous archi-
tecture designed for concurrency.
Modules:
- importing built-in modules
https://nodejs.org/dist/latest-v6.x/docs/api/modules.html#modules_core_modules
Control flow (Async tasks, Callbacks):
- Async lib
- Promices
https://strongloop.com/strongblog/promises-in-node-js-an-alternative-to-callbacks/
Package.json:
- structure
- required fields
- dependencies vs devDependencies
https: //docs.npmjs.com/files/package.json
https: //docs.npmjs.com/creating-a-package-json-file
GLOBAL:
- __dirname
- __filename
https: //nodejs.org/docs/latest/api/globals.html#globals_dirname
https: //nodejs.org/docs/latest/api/globals.html#globals_filename
HTTP create server:
- how to create simple server
https://nodejs.org/docs/latest/api/http.html#http_http_createserver_options_requestlistener
Reading and Writing Files:
- fs.open
https://nodejs.org/docs/latest/api/fs.html#fs_fs_open_path_flags_mode_callback
Process:
- process.cwd
https: //nodejs.org/docs/latest/api/process.html#process_process_cwd
https: //www.hacksparrow.com/nodejs/directory-references.html
Express js:
- serving Static Content
- app.engine
- midleware
- request and response
- routing
- working with view
Express.js is a web framework based on the core Node.js http module 1 and Connect 2 components. Those components
are called middleware. They are the cornerstone of the framework’s philosophy, which is configuration over
convention. Some developers familiar with Ruby compare Express.js to Sinatra, which has a very different approach
from the Ruby on Rails framework that favors convention over configuration. In other words, developers are free
to pick whatever libraries they need for a particular project. This approach provides them with flexibility and the
capability to highly customize their projects.
If you have written any serious apps using only the core Node.js modules, you most likely found yourself
reinventing the wheel by constantly writing the same code for similar tasks, such as:
• Parsing HTTP request bodies
• Parsing cookies
• Managing sessions
• Organizing routes with a chain of if conditions based on URL paths and HTTP methods of the
requests
• Determining proper response headers based on data types
• Handling errors
• Extracting URL parameter (e.g., /messages/3233)
Later, you might have created your own libraries to reuse the code, but your libraries would not be as thoroughly
tested as the best community supported libraries. Also, the maintenance would be fully on you and your team. So, my
recommendation is to use the community module if it suites your needs. This advice applies to using small libraries
and web frameworks as well.
Express.js solves these and many other problems. It provides ways to reuse code elegantly and provides a
model-view-controller (MVC)-like structure for your web apps. The model (M) part needs to be supplied by an
additional database-driver library (e.g., Mongoose 3 ). Those apps could vary from barebones, back-end-only
REST APIs to full-blown, full-stack, real-time web apps with additional libraries such as jade-browser
(https://npmjs.org/package/jade-browser) and socket.io (http://socket.io).
How Express.js Works
Express.js is a node package manager (NPM or npm) module that is a dependency to your application. This means
that every project that is built with/on Express.js needs to have the framework’s source files in the local node_modules
folder (not globally!). For this, you install Express.js just like any other NPM module, with $ npm install, e.g., $ npm
install express@4.2.0.
Now, we can overview a typical structure of an Express.js application. Let’s say your application is in a server.js file
and you plan to start your application with $ node server.js. In that case, you need to require and configure Express.js
in the server.js file. This file usually contains statements to accomplish the following:
1. Include third-party dependencies as well as your own modules such as controllers,
utilities, helpers, and models
2. Instantiations of the Express.js object and others
3. Connect to databases such as MongoDB 4 , Redis 5 , or MySQL 6
4. Configure Express.js app settings such as the template engine and its files’ extensions
5. Define middleware such as error handlers, static files folder, cookies, and other parsers
6. Define routes and their request handlers
7. Start the app which will start the server on a particular host and port
Of course, if your application is big, you’ll have multiple files to configure your Express.js app, not just a single
server.js or app.js file. The reason for that is better code organization. For example, in one file you will configure
sessions, in another authentication, in another routes, and so forth.
Instantiations
To use Express.js, you need to instantiate it. At the same time, it’s a good practice to instantiate any other objects:
var app = express();
var db = mongo.db(‘mongodb://localhost:27017/integration_tests’, {native_parser: true});
Configuring Express.js App Settings
Simply put, configuring Express.js app settings consists of setting some values to string keys with app.set(). Some of
these keys are used by Express.js and augment its behavior, while others are arbitrary. For example, if you are using
the Jade template engine and *.jade files, use ‘view engine’ to let Express.js know that it needs to looks for
*.jade files:
app.set(‘view engine’, ‘jade’);
You’ll find a lot more information about configuring settings in the following chapters.
Sometimes we want to store a custom value on the server object for future reference. For example, we can assign
port to a value from the environment variable PORT or, if it’s undefined, to 3000 so that we can use the value across all
source code:
app.set(‘port’, process.env.PORT || 3000);
Defining Middleware
Middleware is a special function that allows for better code organization and reuse. Some of the middleware is
packaged as third-party (NPM) modules and can be used right out of the box. Other times, we can write our own
custom middleware. In both cases the syntax is app.use():
app.use(bodyParser.json());
Defining Routes
Routes can be either good old web pages or REST API endpoints. In both cases the syntax is similar: we use
app.VERB(), where VERB() is an HTTP method such as GET, POST, DELETE, PUT, OPTIONS, or PATCH. For example,
we can define home page (root) routes as
app.get(‘/’, renderHomePage);
When an Express.js app is running, it listens to requests. Each incoming request is processed according to a
defined chain of middleware and routes, starting from the top and proceeding to the bottom. This aspect is important
because it allows you to control the execution flow.
For example, we can have multiple functions handling each request, and some of those functions will be in the
middle (hence the name middleware):
1. Parse cookie information and go to the next step when done.
2. Parse parameters from the URL and go to the next step when done.
3. Get the information from the database based on the value of the parameter if the user is
authorized (cookie/session), and go to the next step if there is a match.
4. Display the data and end the response.
