Monday, February 11, 2019

error TS5014: Failed to parse file

Problem:
Working on rebuilding my js files from Typescript src files, I went to run tsc and found this issue.

$ tsc -p tsconfig.json

error TS5014: Failed to parse file 'tsconfig.json/tsconfig.json': Unexpected token u in JSON at position 0.


Solution:
Turns out this was due to not having typescript installed on WSL (where I was running the compiler command).

$ npm i -g typescript
$ tsc -p tsconfig.json

By first installing typescript, I was able to properly compile my code.

Initially, I had thought this might be related to using WSL in conjunction with Windows to develop. However, this was not the case. You can read my previous post to see issues related to WSL and Windows cross development.

Nodejs Development - WSL and Windows: Unknown error lstat, errno 4094

I really like Linux and MacOS over Windows. The commands and syntax feel more comfortable, the tooling more reliable (IMO), probably because I've managed Linux servers for some time. So, you can imagine how excited I was to see WSL or Windows Subsystem Linux.

Problem: 
Switching between WSL Ubuntu terminal and Windows command line, I've found some various issues. More recently, however, I came across this one while trying to run a simple script:

{ Error: UNKNOWN: unknown error, lstat 'C:\project_path\node_modules\.bin\eslint'
  errno: -4094,
  code: 'UNKNOWN',
  syscall: 'lstat',
  path: 'C:\\project_path\\node_modules\\.bin\\eslint' }


Solution:
In my case, the problem has to do with how npm is connecting dependency scripts to my project. In WSL, npm uses symbolic links to create an alias to something located somewhere else; in our case the scripts for these programs are actually nested deeper in the node_modules directory. To allow our program to use them easily, npm creates aliases in the node_modules/.bin directory. However, in Windows, npm creates a bunch of cmd files, which more or less do the same thing, by running the script where it actually resides; deeper in the node_modules/ directory.

To figure this out, I decided to let the operating systems each tell me what it saw in the .bin/ directory (I had run npm install on WSL prior):

On WSL (using ls -l)
eslint -> ../eslint/bin/eslint.js

On Windows (using attrib)
The target of the symbolic link C:\project_path\node_modules\.bin\eslint does not exist

If you want to develop node applications, then you either need to use npm on WSL or Windows, but not both. As my IDE is a Windows application and has built in script execution, I guess I will need to do all my npm management on Windows.

To fix this issue, the easiest/clean solution is to completely remove node_modules/ (verify it is removed on both Windows and WSL!) and then on your chosen operating system (Windows for me), run your npm install command.

Wednesday, January 9, 2019

LTI Signature Fails - Carriage Returns, Newlines, and OAuth Signatures

Problem: 

While writing an LTI integration tool, I decided to run a battery of true LTI payloads from various LMS systems and see if I could produce the same signature. While running these payloads via automation, I found that some payloads would fail to produce the same signature. After ensuring that my encoding was using the agreed standard RFC 3986, I looked to narrow in on a specific LMS like Moodle or Blackboard. The signature fails were not confined to a specific LMS, but rather all contained the same single non-visible character.


Solution:

The payloads failing to produce a correct signature happened to contain a carriage return (\r) followed by a new line(\n). The \r\n line termination sequence is commonly used by Microsoft. Suspicious of the carriage return, I removed it from the payloads, then reprocessed the signatures. It was successful. So now understanding what caused the problem, I decided to narrow in to the why.

LTI Flow Inner Workings (The Quick Version)
The LTI flow originates from the LMS also known as the Tool Consumer (TC), the TC will generate a payload of data consisting of key/value pairs and then sign the data using a shared secret. This signature uses OAuth. Once the data has been created it is generally injected into a user's browser page inside a form element of type POST and an action pointing to the Tool Provider's (TP) endpoint. Once this data is in the form, it will be submitted either automatically or manually. The recipient, which is the TP, takes this payload of data and verifies the data has not been changed by using the shared secret to recreate the signature. If the signature matches the one provided, the data can be trusted.

Dirty Little Microsoft Browser
The problem we see is probably a result of the browser attempting to format the data as it is injected into the form and newlines are preceded by a carriage return. This actually makes sense if the browser is going to display the data to the user, as Windows uses these carriage returns.

At some point, it appears that the issue was corrected for Edge, but current LTI payloads make me suspect that it is still an issue. (See: https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/16221536/)

Coding Defensively
Unfortunately, we can't just always remove carriage returns prior to signing our payload. This is because a lot of data in the payload is, in fact, entered by an user who might be using a browser that uses carriage returns. In this case, the data will most likely be signed with the carriage returns in place as few implementations clean up these returns prior to signing (but probably should).

In order to handle these carriage return injections, we are going to have to check both possibilities when evaluating the signature. First, we assume that the data provided is correct for signing. The payloads may not contain carriage returns, may have been signed with the carriage returns, or there might not be any newlines at all. If this fails, we second try to remove only carriage returns that are immediately followed by a newline character.