Transferring data to API from SQL Server to MongoDB using Nodejs and GraphQL
As a developer is harder to catch up with all new frameworks that are coming out, but actually, learning new tools helps to make the development process faster and smoother once start to get comfortable with all of them, when I stopped to have favoritism for some specific framework my life changed completed, if we don’t have favorite is easier to move on and go out of our comfort zone and those days the framework who is trending now it won’t be next year, but the truth is all new trending technology always have something in common all of then only are trying to make our life easy as a developer.
In this post, I want to highlight some issues that I experimented on in my learning process.
Explanation of what I’m trying to do.
I wanted to create a RESTful API able to hold certain data from multiples small businesses like products and clients. Those small businesses might have a database like SQL Server, PostgreSQL, Oracle, Mysql, and so on, after fetching all data I needed to transfer it to the RESTfull API and the API will use MongoDB I choose a not SQL database for the API for the flexibility to add a field to my schema because all clients might have additionals fields on Clients and Product tables.
Technologies for the API:
Client Synchronization
Issues Faced
- GraphQl API secure with an authentication system
- Installing Apollo in nodejs
- NodeJS parses objects using a single quote and date to GraphQL
- Large payload
-
GraphQl API secure with an authentication system
GraphQL doesn’t provide authentication after I made my API works everyone is able to access my data, To solve this issue I implemented a middleware to my express server.
Using jasonwebtoken I was able to generate a token after the user login and my middleware only allow access to Login and Register resources and to access my other resources the request should come with the tokenId.
const jwt = require("jsonwebtoken");
const gql = require("graphql-tag");
const MODULE_EXCLUDE = ["login", "register"];
const isException = req => {
const query = gql`
${req.body.query}
`;
const requestParam =
query.definitions[0].selectionSet.selections[0].name.value;
return requestParam && MODULE_EXCLUDE.includes(requestParam);
};
module.exports = async (req, res, next) => {
try {
if (!isException(req)) {
const toke = req.headers.authorization.split(" ")[1];
const decoded = await jwt.verify(toke, process.env.JWT_KEY);
req.userData = decoded;
} else {
req.userData = null;
}
} catch (error) {
return res.status(401).json({
message: "Auth failed"
});
}
next();
};
On the express server
if (process.env.NODE_ENV !== "development") {
app.use(checkAuth); //<== authentication middleware
}
//after authentication checked
app.use(
"/api",
graphqlHttp(req => ({
schema,
context: {
userData: req.userData //<== req.userData comming from checkAuth
},
rootValue,
graphiql: true
}))
);
After the request authentication is completed then express-graphql allows us to pass data using the context property to all GraphQL functions.
I’m using the development environment variable to disable the authentication because of GraphQL UI tools don’t have any option to pass the header and the UI won’t be accessible. I’m also found a GraphQL UI tool that allows us to make the request including the header GraphQL on Mac
2. Installing Apollo in nodejs
It was my first time using Apollo and the documentation makes this framework very straightforward to use, but I want to include it in my nodejs project, for some reason I tried but didn’t work so I ended up using Babel to use ES6 Import syntax.
const ApolloClient = require('apollo-boost');
const client = new ApolloClient({ url: '');
Error: ApolloClient is not a constructor
/workspace/scratch/temp/test.js:3
const client = new ApolloClient();
^
TypeError: ApolloClient is not a constructor
at Object.<anonymous> (/workspace/syc/temp/test.js:3:16)
at Module._compile (module.js:571:32)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:488:32)
at tryModuleLoad (module.js:447:12)
at Function.Module._load (module.js:439:3)
at Module.runMain (module.js:605:10)
at run (bootstrap_node.js:420:7)
at startup (bootstrap_node.js:139:9)
at bootstrap_node.js:535:3
I tried several approaches using the required.
After installing Babel I was able to use the import and everything works fine
npm install babel-cli babel-preset-env -d --save
Create .babelrc
{
"presets": ["env"]
}
"start": "./node_modules/.bin/babel-node src/app.js"
Then using the import
import ApolloClient from 'apollo-boost';
const client = new ApolloClient({
uri: 'https://graphql.example.com'
});
3. NodeJS parses objects using a single quote
This issue was the most annoying one, I elaborate the query to SQL Server to match with the same Mongo and GraphQL Schema, so the process is like this:
SQL Server Query => Apolly => GraphQL but something was wrong,
I realized the array of the object coming from SQL SERVER has single quotes and GraphQL doesn’t support it.
[ { code: 'AB-0001',
barCode: '',
name: 'PRODUCT 1',
classification: '',
lastPurchase: 2000-01-01T00:00:00.000Z,
price1: 7.5,
price2: 8.2373,
price3: 0,
price4: 4,
price5: 7.4153,
price6: 0,
saleUnit: '',
stock: 1300 },
{ code: 'KO-0003',
barCode: '',
name: 'PRODUCT 2',
classification: 'TQ',
lastPurchase: null,
price1: 3448.305084,
price2: 3831.4500933333297,
price3: 0,
price4: 0,
price5: 3448.305084,
price6: 0,
saleUnit: 'TQ',
stock: 0 } ]
I noticed the output of node.js always uses a single quote no matter if your object doesn’t have
//my object
{
name: "name"
}
node.js output
{
name: 'name'
}
I ended up using stringify-object to convert the single quote to double.
const pretty = stringifyObject(obj, {
indent: ' ',
singleQuotes: false
});
After using the stringify I got rid of the single quote but then I found another bug related to the Date field.
[
{
code: "AB-0001",
barCode: "",
name: "PRODUCT 1",
classification: "",
lastPurchase: new Date('2000-01-01T00:00:00.000Z'), //<== DATE
price1: 7.5,
price2: 8.2373,
price3: 0,
price4: 4,
price5: 7.4153,
price6: 0,
saleUnit: "",
stock: 1300
}
]
To fix this issue I folk this project and I made a small adjustment for that.
if (input instance of Date) {
- return `new Date('${input.toISOString()}')`;
+ return `"${input.toISOString()}"`;
}
then I install that dependency using my account
npm install git+https://github.com/victors1681/stringify-object.git --save
4. Large payload
Everything was working fine but at the moment when I fetch 4,818 rows 1.95 MB from SQL Server and send it to my API I got the following error
Error: request entity too large
//Express Fixed
const app = express();
app.use(bodyParser.json({ limit: "50mb" }));
app.use(bodyParser.urlencoded({ limit: "50mb", extended: true }));
Done!
victors1681/mseller-api
Transferring data to API from SQL Server to MongoDB using Nodejs and GraphQL – victors1681/mseller-api