Wow, amazing! Wooo! I can't wait to polish off the rough edges and play it in Clojure myself.
Yeah, some of the levels have non-writable JavaScript built in; I hadn't thought about that. We'll have to make the user language preference only apply to user-writable code, and construct the Aether instances using JavaScript for the rest of them. Planning on doing things like that this week for the multiplayer anyway. Don't worry about those levels for now.
Basically, it only runs in a web worker, tightly integrated into all of the game engine code. You can access the web worker through the Chrome dev tools, but it's not convenient, and setting a breakpoint in the dynamically compiled user code will be... icky. What are you trying to do with it? Perhaps we can get the time-travel hover debugger built into CodeCombat to help you out.
The separate calls to fetch Aether and Lodash are new (they used to be compiled into world.js), so it's very possible that there are bugs with it. I'm not seeing them now, though. Does it work in Firefox?
If you want, you can stop by our HipChat and we can do this a little more synchronously, since I don't check my email that often: http://www.hipchat.com/g3plnOKqa
I put it in the clojure branch on the main Aether repo. I tried playing CodeCombat with it, but it didn't seem to compile anything (or throw an error). I just realized I never did anything with the zeroing of node ranges thing you had in your branch, so maybe that has something to do with it?
I also added Clojure to the available languages in the latest CodeCombat master if you open up your editor config from the settings gear in the playback bar while playing.
One thing I noticed was that it adds ~400KB of minified JavaScript to the final Aether runtime (which is already huge) and makes the Grunt Uglify step take like 10 minutes. I wonder if there's a way we can improve that.
Yes I had to set the node ranges to 0, 0 because the parser doesn't emit source locations in that format, it uses the standard Mozilla parser format ({ start: { line: ..., column: ... }, end: { line: ..., column: ... } }). It should be easy to convert to the range format (right?), I'll work on that once the more important items are done.
About the size of the parser + library, it's mostly because I wasn't able to figure out how to use Browserify properly, so there are redundant dependencies in there. Here's the list of dependencies:
- parser: estraverse
- assertions: lodash, mori (a library that makes Clojure data-structures available in JS)
- core: lodash, mori, assertions
So the only new things are:
- parser (70 KB pre-minification)
- assertions (5 KB pre-minification)
- core (21 KB pre-minification)
- mori (145 KB, this is already minified)
So in theory the final overhead should be ~ 200 KB (still a lot, but it cannot be reduced further, I think).
The uglify step takes so long because I guess it's trying to minify mori, which is obviously of no use. The ideal way to do it would be to browserify and uglify the parser, assertions, and core library, and then concatenate mori with them. Could you help me out with this?
Perhaps you're wondering why we need to depend on a huge library like Mori, why can't we just implement Clojure's data structures ourselves with less code?
The answer is: those data structures are at the heart of Clojure. They have some important properties, the most significant one being that they're immutable and yet efficient. Read more here: http://clojure.org/data_structures#Data%20Structures.
I could either support these language features or keep the size of the runtime low. The former seemed like the right choice for CodeCombat.
From Aether's point of view, it doesn't need Closer to be minified, because it's going to minify it anyway. Maybe trying to re-minify the minified code is taking a long time? Or maybe because mori and lodash are in there twice? The important thing for us is to figure out how to fix the minification time. I mean, Aether is normally not very efficient and includes lots of stuff, so it's already 1.4MB minified. (I was hoping to reduce that instead of increase it, but whatever.) But it still used to take a few seconds to minify, and now it's almost unbearable.
Maybe you can provide minified and non-minified versions? Many projects that would include this have their own minification step already. I am (along with many) am already pulling in Lo-Dash, so that could possibly be a dependency instead of included (especially if your use is compatible with Underscore, too.
We can also reduce the minification time by loading Mori separately into CodeCombat when the user selects Clojure. Meanwhile I've added non-minified versions to Closer, but that doesn't seem to affect the minification time.
Is there a problem with using while (true) { if ... else break; } on CodeCombat? When I write code like that, clicking the "Cast" button does nothing at all. I need to be able to execute that because Closer parses loop / recur forms (Clojure's looping primitives) into that kind of code.
Hmm, I'm not seeing the problem. Do you have an up-to-date copy of the Inventory System? You can go to the level editor, click Systems, click Inventory, use the menu in the upper right to get its version history, and see which version you have:
assertions.arity = function (expected, args) {
console.log(args); // { 0: 2 }, everything seems ok
console.log(args.length); // undefined! WTH?
}
And calling Array.prototype.slice.call(arguments) before passing it on to assertions.arity merely returns an empty array. Really weird stuff. What do I do?
Hey, I was just thinking, could it cause problems if some of my AST nodes are actually the same object in memory? So if you manipulate one of them all the other nodes will get affected, which could lead to some spooky-action-at-a-distance type weirdness.
I'm done running all the levels except the ones with read-only JS and the multiplayer levels. For Gridmancer I ran just the naive solution. So now what do we do about the remaining levels?
I've fixed the ones that had read-only JS, so those should work now as well–let me know if you still see problems. And actually, there may be a couple bugs, but Michael has just finished remogrifying everything so that multiplayer games only send out your transpiled code to your opponents, so it shouldn't matter if that was originally in JS or not–it may be able to work on the ladders now. (If it doesn't, I'll fix whatever bugs pop up this weekend.)
So there are still issues with documentation all being in JS and the spell editor not switching to the default code for the language you chose, but those can also come later when we do a big "war of the languages" blog post announcing all this. :)
One thing that would be awesome would be if you could submit patches that include the sample code translated to Clojure for at least some of the levels, since I probably won't write good sample code (until I can learn Clojure from CodeCombat, that is). Here's how:
Bubble Sort Bootcamp Battle is unplayable in Clojure. I coded up Tharin, but the soldiers aren't under my control and they throw runtime Programmable problems in the hear() method ("Line 20: Cannot read property 'pos' of undefined").
One problem is that in plan() based levels you can't call "blocking" functions inside any function but plan. (Well you can call them, but they wount block, and the basicly only the last one executed will have an effect). There is an issue to fix this, but is (like all of plan()) is pretty tricky. There is a really simple iterative solution to the towers of hanoi however; but I woun't spoil it.
47 comments
nwinter Manager • over 9 years ago
Wow, amazing! Wooo! I can't wait to polish off the rough edges and play it in Clojure myself.
Yeah, some of the levels have non-writable JavaScript built in; I hadn't thought about that. We'll have to make the user language preference only apply to user-writable code, and construct the Aether instances using JavaScript for the rest of them. Planning on doing things like that this week for the multiplayer anyway. Don't worry about those levels for now.
There are some details on how players' code is run here: https://github.com/codecombat/codecombat/wiki/World
Basically, it only runs in a web worker, tightly integrated into all of the game engine code. You can access the web worker through the Chrome dev tools, but it's not convenient, and setting a breakpoint in the dynamically compiled user code will be... icky. What are you trying to do with it? Perhaps we can get the time-travel hover debugger built into CodeCombat to help you out.
The separate calls to fetch Aether and Lodash are new (they used to be compiled into world.js), so it's very possible that there are bugs with it. I'm not seeing them now, though. Does it work in Firefox?
If you want, you can stop by our HipChat and we can do this a little more synchronously, since I don't check my email that often: http://www.hipchat.com/g3plnOKqa
nwinter Manager • over 9 years ago
I wanted to try it out, so I messed around with it here:
https://github.com/codecombat/aether/commit/bb11330b06959f8cc308416250e9d5e1991bbb16
I put it in the clojure branch on the main Aether repo. I tried playing CodeCombat with it, but it didn't seem to compile anything (or throw an error). I just realized I never did anything with the zeroing of node ranges thing you had in your branch, so maybe that has something to do with it?
I also added Clojure to the available languages in the latest CodeCombat master if you open up your editor config from the settings gear in the playback bar while playing.
One thing I noticed was that it adds ~400KB of minified JavaScript to the final Aether runtime (which is already huge) and makes the Grunt Uglify step take like 10 minutes. I wonder if there's a way we can improve that.
Vicky Chijwani • over 9 years ago
Yes I had to set the node ranges to 0, 0 because the parser doesn't emit source locations in that format, it uses the standard Mozilla parser format ({ start: { line: ..., column: ... }, end: { line: ..., column: ... } }). It should be easy to convert to the range format (right?), I'll work on that once the more important items are done.
About the size of the parser + library, it's mostly because I wasn't able to figure out how to use Browserify properly, so there are redundant dependencies in there. Here's the list of dependencies:
- parser: estraverse
- assertions: lodash, mori (a library that makes Clojure data-structures available in JS)
- core: lodash, mori, assertions
So the only new things are:
- parser (70 KB pre-minification)
- assertions (5 KB pre-minification)
- core (21 KB pre-minification)
- mori (145 KB, this is already minified)
So in theory the final overhead should be ~ 200 KB (still a lot, but it cannot be reduced further, I think).
The uglify step takes so long because I guess it's trying to minify mori, which is obviously of no use. The ideal way to do it would be to browserify and uglify the parser, assertions, and core library, and then concatenate mori with them. Could you help me out with this?
Vicky Chijwani • over 9 years ago
Perhaps you're wondering why we need to depend on a huge library like Mori, why can't we just implement Clojure's data structures ourselves with less code?
The answer is: those data structures are at the heart of Clojure. They have some important properties, the most significant one being that they're immutable and yet efficient. Read more here: http://clojure.org/data_structures#Data%20Structures.
I could either support these language features or keep the size of the runtime low. The former seemed like the right choice for CodeCombat.
nwinter Manager • over 9 years ago
From Aether's point of view, it doesn't need Closer to be minified, because it's going to minify it anyway. Maybe trying to re-minify the minified code is taking a long time? Or maybe because mori and lodash are in there twice? The important thing for us is to figure out how to fix the minification time. I mean, Aether is normally not very efficient and includes lots of stuff, so it's already 1.4MB minified. (I was hoping to reduce that instead of increase it, but whatever.) But it still used to take a few seconds to minify, and now it's almost unbearable.
Maybe you can provide minified and non-minified versions? Many projects that would include this have their own minification step already. I am (along with many) am already pulling in Lo-Dash, so that could possibly be a dependency instead of included (especially if your use is compatible with Underscore, too.
For splitting out Mori, maybe something like this: https://github.com/codecombat/aether/blob/master/Gruntfile.coffee#L79-L82
Vicky Chijwani • over 9 years ago
The size is going up by 400 KB because the core library gets included twice: once when requiring it in clojure.coffee (https://github.com/vickychijwani/aether/blob/clojure/src/languages/clojure.coffee#L3), and again when loading it via fs.readFileSync in Aether (https://github.com/vickychijwani/aether/blob/clojure/src/aether.coffee#L223). This will not happen once the runtimeGlobals option is in place. Is that ready yet?
We can also reduce the minification time by loading Mori separately into CodeCombat when the user selects Clojure. Meanwhile I've added non-minified versions to Closer, but that doesn't seem to affect the minification time.
nwinter Manager • over 9 years ago
Yeah, I hooked up the runtimeGlobals in the clojure branch of the main Aether repo and removed the readFileSync part.
Vicky Chijwani • over 9 years ago
Is there a problem with using while (true) { if ... else break; } on CodeCombat? When I write code like that, clicking the "Cast" button does nothing at all. I need to be able to execute that because Closer parses loop / recur forms (Clojure's looping primitives) into that kind of code.
Vicky Chijwani • over 9 years ago
I'm unable to load Coin Mania on my setup anymore, in either Firefox or Chrome, whether logged in or not.
The error in Firefox is:
TypeError: this.thang.world.initialTeamGold is undefined app.js:8901.
And in Chrome it is:
TypeError: Cannot read property 'humans' of undefined CocoSprite.coffee:605
module.exports.CocoSprite.updateGold CocoSprite.coffee:605
module.exports.CocoSprite.update CocoSprite.coffee:189
module.exports.SpriteBoss.update SpriteBoss.coffee:168
module.exports.Surface.updateState Surface.coffee:598
module.exports.Surface.showLevel Surface.coffee:439
module.exports.Surface.setWorld Surface.coffee:118
(anonymous function) Surface.coffee:380
p._runActions vendor.js:41143
p.setPosition vendor.js:41018
p.tick vendor.js:41036
Tween.tick vendor.js:40549
Tween.handleEvent vendor.js:40564
p._dispatchEvent vendor.js:18291
p.dispatchEvent vendor.js:18216
Ticker._tick vendor.js:19040
Ticker._handleSynch vendor.js:18969
Any idea what's causing this?
nwinter Manager • over 9 years ago
Hmm, I'm not seeing the problem. Do you have an up-to-date copy of the Inventory System? You can go to the level editor, click Systems, click Inventory, use the menu in the upper right to get its version history, and see which version you have:
https://www.dropbox.com/s/m72dqehvbvue8kp/Screenshot%202014-05-14%2010.18.42.png
Is it 0.12?
Vicky Chijwani • over 9 years ago
It's 0.11. How do I update it?
Vicky Chijwani • over 9 years ago
Say I have some transpiled code like this:
function blah (x) {
console.log(arguments); // outputs { 0: 2 }
console.log(arguments.length); // outputs 1
assertions.arity(1, arguments);
...
}
blah(2);
Inside assertions.arity:
assertions.arity = function (expected, args) {
console.log(args); // { 0: 2 }, everything seems ok
console.log(args.length); // undefined! WTH?
}
And calling Array.prototype.slice.call(arguments) before passing it on to assertions.arity merely returns an empty array. Really weird stuff. What do I do?
Vicky Chijwani • over 9 years ago
So I pulled in the latest dump, but the Inventory System didn't get updated. It might be because I didn't clear my local db first. How do I do that?
Vicky Chijwani • over 9 years ago
Hey, I was just thinking, could it cause problems if some of my AST nodes are actually the same object in memory? So if you manipulate one of them all the other nodes will get affected, which could lead to some spooky-action-at-a-distance type weirdness.
Vicky Chijwani • over 9 years ago
I'm done running all the levels except the ones with read-only JS and the multiplayer levels. For Gridmancer I ran just the naive solution. So now what do we do about the remaining levels?
nwinter Manager • over 9 years ago
I've fixed the ones that had read-only JS, so those should work now as well–let me know if you still see problems. And actually, there may be a couple bugs, but Michael has just finished remogrifying everything so that multiplayer games only send out your transpiled code to your opponents, so it shouldn't matter if that was originally in JS or not–it may be able to work on the ladders now. (If it doesn't, I'll fix whatever bugs pop up this weekend.)
So there are still issues with documentation all being in JS and the spell editor not switching to the default code for the language you chose, but those can also come later when we do a big "war of the languages" blog post announcing all this. :)
One thing that would be awesome would be if you could submit patches that include the sample code translated to Clojure for at least some of the levels, since I probably won't write good sample code (until I can learn Clojure from CodeCombat, that is). Here's how:
1. Go to the level editor: http://codecombat.com/editor/level/rescue-mission
2. Double-click the Programmable Thang to find its Programmable Component and the default code
3. Add a "languages" field parallel to the "source" field if it's not there, and add a "clojure" subfield with the sample code translated to Clojure: https://www.dropbox.com/s/mh30qvkz9v91a7o/Screenshot%202014-05-17%2007.24.29.png
Vicky Chijwani • over 9 years ago
Awesome, I'll get right on it. I'll add the sample codes as well.
Vicky Chijwani • over 9 years ago
I'm having some trouble completing Ogres of Hanoi. Recursion doesn't seem to work properly on CoCo. I tried both Clojure and JavaScript. What's wrong with this solution? https://gist.github.com/vickychijwani/44a2e31d1cb434cc6159
Vicky Chijwani • over 9 years ago
Bubble Sort Bootcamp Battle is unplayable in Clojure. I coded up Tharin, but the soldiers aren't under my control and they throw runtime Programmable problems in the hear() method ("Line 20: Cannot read property 'pos' of undefined").
Rob Blanckaert • over 9 years ago
One problem is that in plan() based levels you can't call "blocking" functions inside any function but plan. (Well you can call them, but they wount block, and the basicly only the last one executed will have an effect). There is an issue to fix this, but is (like all of plan()) is pretty tricky. There is a really simple iterative solution to the towers of hanoi however; but I woun't spoil it.
Vicky Chijwani • over 9 years ago
Ah, thanks for clearing that up. I'll try an iterative solution :)
nwinter Manager • over 9 years ago
@vickychijwani can you send me an email? nick@codecombat.com