Getting started with node.js, Express and Mongoose

I started tinkering with some test data creation scripts using Node.js a while back, but it’s been on my todo list to dig deeper into some Node. I recently walked through this very good article, and got a REST based backend using Node.js, Express and a MongoDB up and running in less than an hour. Pretty good going for learning a new tech stack 🙂

After putting together the app from the steps in the article, I realized there was a few steps that could have been simplified. For example, instead of hand coding the package.json as in the article, you can let npm update it for you by passing the –save option. So, to create an initial package.json:

npm init

and walk through the prompts.

Next, instead of manually adding the dependencies for Express, Mongoose and body-parser, again, use npm to install and add them to the package.json file with –save:

npm install express --save

npm install mongoose --save

npm install body-parser --save

That saves some work – no need to edit the package.json file by hand when the tool maintains it for you. Without the –save option npm still downloads the dependency, but –save writes the details into your package.json as well.

Using Mongoose with Express certainly gets you up and running with basic CRUD using REST pretty quick and easy with minimal coding. I was surprised how little code it takes to get the basics implemented. I’ve shared my version of the completed app to GitHub for future reference here, and for quick reference below:

[code]

var express = require(‘express’);
var app = express(); //init Express
var bodyParser = require(‘body-parser’);
var mongoose = require(‘mongoose’);
var Contact = require(‘./app/models/Contact’);
mongoose.connect(‘mongodb://nodetest:yourpassword@localhost:27017/nodetest’);
//init bodyParser to extract properties from POST data
app.use(bodyParser.urlencoded({ extended: true }));
app.use(bodyParser.json());

var port = process.env.PORT || 8080;

//init Express Router
var router = express.Router();

//default/test route
router.get(‘/’, function(req, res) {
res.json({ message: ‘App is running!’ });
});

router.route(‘/contacts/:contact_id’)
// retrieve contact: GET http://localhost:8080/api/bears/:bear_id)
.get(function(req, res) {
Contact.findById(req.params.contact_id, function(err, contact) {
if (err)
res.send(err);
res.json(contact);
});
})
// update contact: PUT http://localhost:8080/api/contacts/{id}
.put(function(req, res) {
Contact.findById(req.params.contact_id, function(err, contact) {
if (err) {
res.send(err);
}
else {
contact.firstName = req.body.firstname;
contact.lastName = req.body.lastname;
contact.save(function(err) {
if (err)
res.send(err);

res.json({ message: ‘Contact updated!’ });
})
}
});
})
//delete a contact
.delete(function(req, res) {
Contact.remove({
_id: req.params.contact_id
}, function(err, contact) {
if (err)
res.send(err);

res.json({ message: ‘Successfully deleted contact’ });
});
});

router.route(‘/contacts’)
// create contact: POST http://localhost:8080/api/contacts
.post(function(req, res) {
var contact = new Contact();
contact.firstName = req.body.firstname;
contact.lastName = req.body.lastname;

contact.save(function(err) {
if (err)
res.send(err);

res.json({ message: ‘Contact created!’ });
});
})
//GET all contacts: http://localhost:8080/api/contacts
.get(function(req, res) {
Contact.find(function(err, contacts) {
if (err)
res.send(err);

res.json(contacts);
});
});

//associate router to url path
app.use(‘/api’, router);

//start the Express server
app.listen(port);
console.log(‘Listening on port ‘ + port);
[/code]

Splitting an mpeg4 file into multiple parts with ffmpeg

I have a 5.3GB mpeg4 file (of the just released ‘From Bedrooms to Billions: Amiga Years‘ documentary about the Commodore Amiga – I haven’t watched it yet but may post a review later), and the size of this single file is giving me some issues 🙂

It’s too large for a FAT32 formatted flash drive, since the largest file you can store on that file system is 4GB. It’s also too large to burn to a 4.7GB single-sided DVD. So I could have reformatted a flashdrive to exFAT (I don’t think the PS4 reads NTFS drives, and my Mac won’t write to NTFS, at least not without other drivers), but the easiest option is probably to split the file to smaller parts.

Now, if you’ve done any video editing before, rendering 1080p video and any amount of transcoding to different formats takes a ton of time. I didn’t want to recode the video or change the format in any way, I just wanted to split the file.

Turns out ffmpeg will do exactly this, and to split into 2x 1hr15m smaller files only took a few minutes on my i7 Mac. I took suggestions from this question, and ended up with these two commands:

ffmpeg -i inputfile.mp4 -ss 00:00:00.0 -c copy -t 01:15:00.0 outputfile-pt1.mp4

ffmpeg -i inputfile.mp4 -ss 01:15:00.0 -c copy outputfile-pt2.mp4

The first one takes input from the start and for 1hr15mins writes to the output file, and the second starts from 1hr15min into the input file and writes the remainder from that point onwards to the second output file.

Inline text replacement with sed

Replacing values in files is incredibly easy with sed. Here’s some examples:

 

sed 's/match/replace/g' file.txt

Find match, replace with replace, globally (all matches), in file.txt

 

sed 's/match/replace/g' file.txt > file2.txt

Same as before, but write results to new file, file2.txt

 

sed -n 's/match/replace/p' file.txt

-n suppresses output of the results, but /p prints out just the matching patterns that are replaced.

 

sed -i.old 's/match/replace/g' file.txt

Replace matches in the input file with inline replace (-i), renaming original file file.txt.old and writing inline replace results to file.txt

 

Node.js and Mongoose: connection hanging on find()

Following this tutorial to build a simple REST api on Node.js with Express, attempting to retrieve all docs from my MongoDB gave these errors:

2016-05-19T20:10:47.311-0700 [initandlisten] connection accepted from 127.0.0.1:52053 #4 (2 connections now open)

2016-05-19T20:10:47.325-0700 [conn4]  authenticate db: nodetest { authenticate: 1, user: "", nonce: "xxx", key: "xxx" }

2016-05-19T20:10:47.336-0700 [conn4] Failed to authenticate @nodetest with mechanism MONGODB-CR: ProtocolError field missing/wrong type in received authenticate command

Some quick Googling turned up the same error in this post that just suggested authentication was failing because no user was passed. So created a new used with:

db.createUser({user : 'nodetest', pwd: 'passwordhere', roles: ['readWrite', 'nodetest']})

… and problem solved.