Add Faking the Mongo Eval Command

site/posts/2020-01-28-faking-mongo-eval.md

@@ -0,0 +1,84 @@
+```
+metadata.title = "Faking the Mongo Eval Command"
+metadata.category = "swift"
+metadata.date = "2020-01-28 19:33:42 -0400"
+metadata.shortDesc = "MongoDB 4.2 removed the eval command, which is a good security measure, but unfortunate for building database-viewing GUI."
+metadata.slug = "faking-mongo-eval"
+```
+
+One of the changes in MongoDB 4.2 was the removal of the `eval` command. While a reasonable security measure, this is rather annoying if you're building [an app](https://git.shadowfacts.net/shadowfacts/MongoView) for interacting directly with a Mongo database. If you want to be able to run commands directly on the database, you now have to go through the `mongo` shell. This seems straightforward, but actually getting the data back into a format that's usable is a bit of a hassle.
+
+<!-- excerpt-end -->
+
+Actually running the command is, surprisingly, the easiest part of this whole endeavor. You can simply launch a [`Process`](https://developer.apple.com/documentation/foundation/process) which invokes the `mongo` shell with a few options as well the command to evaluate:
+
+```swift
+let mongoProc = Process()
+process.launchPath = "/usr/local/bin/mongo"
+mongoProc.arguments = ["mongodb://localhost:27017/your_database", "--quiet", "--norc", "--eval", command]
+mongoProc.launch()
+```
+
+The `--quiet` option prevents the shell from logging its own messages, making parsing the output a little easier. The `--norc` option prevents it from executing `.monorc.js` on startup, so that the environment our command is running in is entirely standard. The `--eval` option does exactly what it says, it evaluates the following parameter in the shell.
+
+This bit of code does make the assumption that the mongo shell is installed in or linked to `/usr/local/bin/mongo` (i.e., it's been installed through Homebrew). To do this properly, you would probably want to try and detect where Mongo is installed and use that path, as well as offer the user a way of customizing the path.
+
+One additional thing to note is that launching an arbitrary executable requires either the App Sandbox be disabled, or Full Disk Access be requested (at least on macOS Catalina), otherwise the process will fail to launch with a message saying "launch path not accessible".
+
+Getting the output is a little bit more difficult, but still not too complicated.
+
+```swift
+let outputPipe = Pipe()
+
+let mongoProc = Process()
+// ...
+mongoProc.standardOutput = outputPipe
+mongoProc.launch()
+
+let outputHandle = outputPipe.fileHandleForReading
+
+var output = ""
+var data: Data!
+do {
+	data = outputHandle.availableData
+	output.append(String(data: data, encoding: .utf8))
+} while (data.count > 0)
+
+outputHandle.closeFile()
+```
+
+We can create a [`Pipe`](https://developer.apple.com/documentation/foundation/pipe) object representing a UNIX pipe. The `mongo` process then uses that pipe as its stdout. We can then read from the pipe's output file handle in order to get the contents of what the shell printed.
+
+Many StackOverflow posts on the topic of getting the output from a process just call `waitUntilExit` on the process and then read the entirety of the data from the pipe's output file handle. While suitable for small output, this approach does not work for situations where the total output of the command is greater than the buffer size of the pipe (as may very well be the case when running queries against large databases). To solve this, we need to continuously read from the pipe until there's no remaining data (meaning the pipe has closed).
+
+Now that we've got the output from Mongo, we need to get it into Swift. Unfortunately, parsing it is a bit annoying. The `mongo` shell outputs a non-standardized format that's like JSON, but with a bunch of JavaScript helpers (e.g. `ObjectId("5e00eb48a14888e105a74fda")`) embedded in it. The [MongoSwift](https://github.com/mongodb/mongo-swift-driver) library can't parse this format (nor can anything else, as far as I can tell). So, in order to turn the shell output into the [Extended JSON](https://docs.mongodb.com/manual/reference/mongodb-extended-json/) format that MongoSwift can parse, we'll need to modify the command that we invoke the shell with.
+
+We'll add some helper code at the beginning of the command we send that defines a function on both the `Object` and `Array` prototypes. This function will take whatever it's invoked on, pass it through `JSON.stringify` to convert it to Extended JSON, and then print it to the console.
+
+The same function defined on the `Array` prototype will perform the same operations, just for each operation in the array, instead of on the array object as a whole. This isn't strictly necessary, but for my purposes I don't want to deal with top-level arrays, and this will make handling it a bit simpler as top-level array elements will be newline-delimited.
+
+```javascript
+Object.prototype.printExtJSON = function() { print(JSON.stringify(this)); };
+Array.prototype.printExtJSON = function() { this.map(JSON.stringify).forEach(it => print(it)); };
+```
+
+For the Array helper, we can't just call `.forEach(print)` since `forEach` passes in multiple arguments (the value, the current index, and the whole array) all of which would get printed out if passed directly to `print`.
+
+We can include these helpers at the beginning of our command and call it on the expression we've been passed in (where `prelude` is a string containing the above JavaScript code):
+
+```swift
+let command = "\(prelude)\(command).printExtJSON()"
+```
+
+This approach does have a drawback: only the result of the last expression in the user-inputted `command` will be stringified and printed. The results of any statements before will be lost, unless the command specifically calls our `printExtJSON` helper. Again, for my purposes, this is a reasonable trade off.
+
+Now, back to Swift. We've got the Extended JSON output from the Mongo shell as one giant string and we just need to have MongoSwift parse it into something usable. Because of the way we're printing arrays (separate `print()` calls for each element) and the fact that we've put everything through `JSON.stringify`, it is guaranteed that there will be only one document per line of output. So, to parse each document separately, we can simply split the output we got at newlines and parse each individually:
+
+```swift
+let decoder = BSONDecoder()
+let result = output.components(separatedBy: "\n").compactMap { (json) in
+  try? decoder.decode(BSON.self, from: json)
+}
+```
+
+And there we have it, the data is finally in a form we can understand from the Swift side of things. If you want to see the whole source code for this, it's [part of MongoView](https://git.shadowfacts.net/shadowfacts/MongoView/src/commit/9488c108b693607e827ef77e5bc16f2cdd491f7c/MongoView/MongoEvaluator.swift). As far as I have come up with, that's about the best way of replicating the `eval` command of previous versions of Mongo. If you have any suggestions for how to improve this or make it more robust, let me know!