[Pkg-javascript-devel] Bug#801897: TypeError: Request path contains unescaped characters
Gerald Turner
gturner at unzane.com
Fri Mar 25 19:42:03 UTC 2016
Package: npm
Version: 1.4.21+ds-2
Followup-For: Bug #801897
I get the same backtrace that Stephen reported, and for the same reason
(http_proxy et al. environment variables and/or .npmrc config settings),
so I decided to step through code using the builtin nodejs debugger to
see what's going on.
$ nodejs debug /usr/share/npm/bin/npm-cli.js view npm
…
debug> sb('/usr/lib/nodejs/tunnel-agent/index.js', 117)
…
debug> c
break in /usr/lib/nodejs/tunnel-agent/index.js:117
115
116 debug('making CONNECT request')
>117 var connectReq = self.request(connectOptions)
118 connectReq.useChunkedEncodingByDefault = false // for v0.6
119 connectReq.once('response', onResponse) // for v0.6
debug> repl
Press Ctrl + C to leave debug repl
> connectOptions
{ host: 'proxy.local',
port: 3128,
proxyAuth: null,
headers: { Host: 'registry.npmjs.org:443' },
method: 'CONNECT',
path: '[object Object]:undefined',
agent: false }
>
There's the culprit, connectOptions.path = '[object Object]:undefined'
when it should probably be 'registry.npmjs.org:443'.
The connectOptions variable is created earlier in this function
(createSocket) with the following code at line 104:
var connectOptions = mergeOptions({}, self.proxyOptions,
{ method: 'CONNECT'
, path: options.host + ':' + options.port
, agent: false
}
)
The options variable, which is referenced when assigning the path
member, is an argument passed into this function call. Back in the
debugger, I examine the options.host and options.port members:
> options.host
{ path: '/npm',
strictSSL: true,
headers:
{ 'npm-session': 'fd065128873578be',
version: '1.4.21',
referer: 'view npm',
accept: 'application/json',
'accept-encoding': 'gzip',
'user-agent': 'npm/1.4.21 node/v4.3.1 linux x64',
host: 'registry.npmjs.org' },
proxy:
{ protocol: 'http:',
slashes: true,
auth: null,
host: 'proxy.local:3128',
port: '3128',
hostname: 'proxy.local',
hash: null,
search: null,
query: null,
pathname: '/',
path: '/',
href: 'http://proxy.local:3128/' },
uri:
{ protocol: 'https:',
slashes: true,
auth: null,
host: 'registry.npmjs.org',
port: 443,
hostname: 'registry.npmjs.org',
hash: null,
search: null,
query: null,
pathname: '/npm',
path: '/npm',
href: 'https://registry.npmjs.org/npm' },
redirects: [],
ca: null,
localAddress: undefined,
__isRequestRequest: true,
agent:
{ options: { proxy: [Object], rejectUnauthorized: undefined, ca: null },
proxyOptions:
{ host: 'proxy.local',
port: 3128,
proxyAuth: null,
headers: [Object] },
maxSockets: 'Infinity',
requests: [],
sockets: [ {} ],
_events: { free: [Function] },
_eventsCount: 1,
request: [Function],
createSocket: [Function] },
_eventsCount: 4,
method: 'GET',
setHost: true,
_maxListeners: undefined,
httpModule:
{ Server: [Function],
createServer: [Function],
globalAgent:
{ domain: null,
_events: [Object],
_eventsCount: 1,
_maxListeners: undefined,
defaultPort: 443,
protocol: 'https:',
options: [Object],
requests: {},
sockets: {},
freeSockets: {},
keepAliveMsecs: 1000,
keepAlive: false,
maxSockets: 'Infinity',
maxFreeSockets: 256,
maxCachedSessions: 100,
_sessionCache: [Object] },
Agent: [Function],
request: [Function],
get: [Function] },
originalCookieHeader: undefined,
cert: null,
followRedirect: true,
encoding: null,
callback: [Function],
_jar: undefined,
pool: {},
dests: [],
port: 443,
followAllRedirects: false,
clientErrorHandler: [Function],
href: 'https://registry.npmjs.org/npm',
readable: true,
writable: true,
domain: null,
_started: true,
_events:
{ error: [ [Function], [Function] ],
complete: [Function],
pipe: [Function],
socket: [Function] },
host: 'registry.npmjs.org',
_defaultAgent:
{ domain: null,
_events: { free: [Function] },
_eventsCount: 1,
_maxListeners: undefined,
defaultPort: 443,
protocol: 'https:',
options: { path: null },
requests: {},
sockets: {},
freeSockets: {},
keepAliveMsecs: 1000,
keepAlive: false,
maxSockets: 'Infinity',
maxFreeSockets: 256,
maxCachedSessions: 100,
_sessionCache: { map: {}, list: [] } },
explicitMethod: true,
tunnel: true,
_redirectsFollowed: 0,
_disableCookies: true,
maxRedirects: 10,
key: null,
_callback: [Function],
_parserErrorHandler: [Function] }
> options.port
>
That explains the '[object Object]:undefined' value of the path member
since options.host is a large object and options.port is undefined. So
I restart the debugger and set a breakpoint earlier to the preceding
line in the backtrace:
$ nodejs debug /usr/share/npm/bin/npm-cli.js view npm
debug> sb('/usr/lib/nodejs/tunnel-agent/index.js', 184)
debug> c
break in /usr/lib/nodejs/tunnel-agent/index.js:184
182 function createSecureSocket(options, cb) {
183 var self = this
>184 TunnelingAgent.prototype.createSocket.call(self, options, function(socket) {
185 // 0 is dummy port for v0.6
186 var secureSocket = tls.connect(0, mergeOptions({}, self.options,
debug> repl
> options.host
{ path: '/npm',
…
> options.port
>
Here the options.host member is still that large object and options.port
is undefined. So I restart the debugger once more with the preceding
line in the backtrace:
$ nodejs debug /usr/share/npm/bin/npm-cli.js view npm
debug> sb('/usr/lib/nodejs/tunnel-agent/index.js', 80)
debug> c
break in /usr/lib/nodejs/tunnel-agent/index.js:80
78
79 // If we are under maxSockets create a new one.
>80 self.createSocket({host: host, port: port, request: req}, function(socket) {
81 socket.on('free', onFree)
82 socket.on('close', onCloseOrRemove)
debug> repl
> host
{ path: '/npm',
…
> port
>
Interesting - the options object is created here, host and port members
reference the arguments passed into this function (addRequest), which
yet again are the large object and undefined respectively. The function
prototype is defined with the following code:
TunnelingAgent.prototype.addRequest = function addRequest(req, host, port) {
var self = this
if (self.sockets.length >= this.maxSockets) {
// We are over limit so we'll add it to the queue.
self.requests.push({host: host, port: port, request: req})
return
}
// If we are under maxSockets create a new one.
self.createSocket({host: host, port: port, request: req}, function(socket) {
…
I get the feeling that addRequest is being called from some internal
nodejs library since the next couple frames of the backtrace have no
apparent source code:
npm ERR! at TunnelingAgent.addRequest (/usr/lib/nodejs/tunnel-agent/index.js:80:8)
npm ERR! at new ClientRequest (_http_client.js:131:16)
npm ERR! at Object.exports.request (http.js:31:10)
npm ERR! at Object.exports.request (https.js:167:15)
npm ERR! at Request.start (/usr/lib/nodejs/request/index.js:607:30)
Looking at /usr/lib/nodejs/request/index.js line 607 yields the
following code:
self.req = self.httpModule.request(reqOptions, self.onResponse.bind(self))
Quite likely matching the documentation for nodejs http.request()¹ API,
however I haven't found any API documentation for addRequest.
Sorry that I haven't found a solution, but I believe I'm onto something,
an API incompatibility perhaps? Also parodon my ignorance - this whole
JavaScript outside the browser is completely foreign to me - and rather
scary, heck one of the stackoverflow answers to this bug was to 'curl
http://some-random-script | sh'² and another work-around is to change
the npm repository URL from https to http³ with no hint about it's
security implications (I get the feeling that TLS is the only defense in
this ecosystem).
¹ https://nodejs.org/api/http.html#http_http_request_options_callback
² http://stackoverflow.com/questions/18006179/typeerror-request-path-contains-unescaped-characters/33216498#33216498
³ http://stackoverflow.com/questions/18006179/typeerror-request-path-contains-unescaped-characters/18012840#18012840
-- System Information:
Debian Release: stretch/sid
APT prefers testing-updates
APT policy: (500, 'testing-updates'), (500, 'testing'), (50, 'unstable')
Architecture: amd64 (x86_64)
Kernel: Linux 4.4.0-1-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to en_US.UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
Versions of packages npm depends on:
ii node-abbrev 1.0.5-2
ii node-ansi 0.3.0-2
ii node-ansi-color-table 1.0.0-1
ii node-archy 0.0.2-1
ii node-block-stream 0.0.7-1
ii node-fstream 0.1.24-1
ii node-fstream-ignore 0.0.6-2
ii node-github-url-from-git 1.1.1-1
ii node-glob 4.0.5-1
ii node-graceful-fs 3.0.2-1
ii node-gyp 3.3.1-2
ii node-inherits 2.0.1-3
ii node-ini 1.1.0-1
ii node-lockfile 0.4.1-1
ii node-lru-cache 2.3.1-1
ii node-minimatch 1.0.0-1
ii node-mkdirp 0.5.0-1
ii node-nopt 3.0.1-1
ii node-npmlog 0.0.4-1
ii node-once 1.1.1-1
ii node-osenv 0.1.0-1
ii node-read 1.0.5-1
ii node-read-package-json 1.2.4-1
ii node-request 2.26.1-1
ii node-retry 0.6.0-1
ii node-rimraf 2.2.8-1
ii node-semver 2.1.0-2
ii node-sha 1.2.3-1
ii node-slide 1.1.4-1
ii node-tar 1.0.3-2
ii node-underscore 1.7.0~dfsg-1
ii node-which 1.0.5-2
ii nodejs 4.3.1~dfsg-3
npm recommends no packages.
npm suggests no packages.
-- no debconf information
--
Gerald Turner <gturner at unzane.com> Encrypted mail preferred!
OpenPGP: 4096R / CA89 B27A 30FA 66C5 1B80 3858 EC94 2276 FDB8 716D
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 948 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-javascript-devel/attachments/20160325/1af7875f/attachment.sig>
More information about the Pkg-javascript-devel
mailing list