Documentation for interop is spread all over the place. Under the "Reference" section:
- "Dependencies": mostly about how to consume javascript libraries
- clojurescript.org: "Packaging Foreign Dependenciees": mostly about how to provide javascript libraries for others to consume
- "Advanced Compilation": mostly duplicative, but also describes how to access cljs vars from javascript
- "JS Module Support": describes an alpha-quality feature from GCC to allow optimized imports of GCC compatible libraries without using
goog.provide
In addition, under "Guides", there is:
- "Externs (Alpha)": describes an alpha-quality feature that apparently infers "externs", or, in other words, infers which symbols GCC should not rename. This document is super confusing. I can't figure out what it actually does or how I'm supposed to use it. There's a variable called
*warn-on-infer*
that really means "warn on failure to infer" I think. - "T. Heller's Improved Externs Inference": this is a bit easier to understand and points out that specific "typing" isn't necessary. Not that I understand what "typing" even means in this context.
The primary example is a library that simply assigns a function to a global:
yayQuery = function() {
var yay = {};
yay.sayHello = function(message) {
console.log(message);
};
yay.getMessage = function() {
return 'Hello, world!';
};
return yay;
};
The structure of this page was not initially obvious. It is divided into "external" code (i.e. code that is loaded using a <script>
tag) and "bundled" code (i.e. code that the cljs compiler will pass along to GCC to bundle into the javascript target)
- The steps are:
- Inline library in a
<script>
tag - Pass
:externs
to cljs compiler to tame GCC - In your code, access the global variable (e.g.
js/yayQuery
)
- Inline library in a
- Write GCC compliant code and import using the
:libs
compiler option - Use the
:foreign-libs
compiler option:- Put the library in a file (or, presumably, copy a UMD build from an npm download)
- Pass this option to the compiler
:foreign-libs [{:file "mylib.js" :provides ["arbitrary-ns"]}]
- Pass
:externs
to cljs compiler to tame GCC - In your code, do a
:require [arbitrary-ns]
, access the global variable (e.g.js/yayQuery
). Obviously you should useyayQuery
instead ofarbitrary-ns
but I just did that to show that the library dictates how you use it whereas the :foreign-libs compiler option dictates how you include it in the target bundle.
- Use cljsjs:
- Either download the jar directly or reference it in your build tool so it downloads it for you (the compiler doesn't need any build options--it just need to be in the classpath presumably)
- Cljsjs jars will contain a
deps.js
file. This file doesn't appear to be documented anywhere except by example in this page, at least, not that google knows about. It appears to be a map that can contain only one key: theforeign-libs
key. (Note, the documentation suggests that you provide a:file
and a:file-min
here. The docs here explain that:file-min
is used with advanced compilations. The point here is that the cljsjs jar contains a pre-built javascript target that presumably sets some global variables, correct externs, and instructions to the compiler so it can find all this stuff.
Note that the only difference between option 2 of the "bundled" options and the "external" version is that you pass the compiler :foreign-libs
, which tells it where the library is and tells it how to know what cljs code is importing it. You don't need the provides statement in the external version because you will just access a global variable at runtime.
Further note that in option 2, you never actually use the arbitrary-ns
outside of the require. One wonders why this is even needed--if you provided the foreign-libs
in your build system presumably you want it bundled. I assume this is to minimize what gets included in the bundle.
This page is almost completely duplicative of option 3 of the above "Bundled" code options. Some odd unexplained thing here:
- The page mentions "your JAR" out of the blue like I know what that is. Aren't we in a javascript environment? I guess cljs uses jars instead of zipfiles. If you are going to write a separate page on packaging foreign dependencies, should you tell me how to do that? Or maybe link to it? I'm sure this is obvious to cljs library providers.
- Again, we are told we should use
:file-min
but not told what really happens with it. - Most confusingly, we are told we must provide transitive dependences in a
:requires
vector. It doesn't actually explain, but from the example it appears that that contents of the:requires
vectors are other synthetic namespaces, just like thearbitrary-ns
from the "Bundled" code example. Given that the examples so far have made themselves available with global variables, I'm not sure what difference it makes to cljs to have this dependency graph. One possibility: cljs doesn't automatically include the:file
s listed inteh:foreign-libs
vector. Instead, perhaps cljs follows the dependency graph starting from therequire
statement in the core code all the way through all foreign libs. That would make sense if the goal was to eliminate unused javascript libraries. - If my supposition in 3 is correct, then maybe the right thing to do is to omit the
:requires
in the case of dependencies that are going to be provided via other mechanisms, such as howreact
is often supplied byreagent
orrum
. - How are you supposed to manage versions with this system? If you have several react libraries, you don't want each of them importing their own copy.
This page isn't about advanced compilation so much as it is about the problems that GCC's name munging will cause with javascript interop.
- Provide an externs file. This file only affects optimized cljs code, not javascript library code. When using an externs file, the library code will be left alone. (Of course, if you write GCC compatible javascript with a
goog.provide
, then everything will be munged. But in that case you won't be using an externs file.) - To access cljs vars from javascript, annotate with
^:export
metadata.
GCC can apparently "convert JavaScript modules into Google Closure modules," meaning you can feed certain javascript libraries and it will somehow optimize the code.
You do this by providing the :module-type
key to the :foreign-libs
compiler option. Its values can be :commonjs
, :amd
, and :es6
.
The example only shows a commonjs example. To use the exports
object from the library, you just :require
the synthetic namespace (e.g. the arbitrary-ns
from above). The properties of the exports
object will be available as vars within the synthetic namespace. (E.g.: (calc/add 4 5)
to access module.exports.add
.)
-
How do you obtain default ES6 exports? Do you just treat the namespace as a var? I assume named es6 exports work as if you did a
import * as MyLib from 'mylib.js'
? -
This apparently will break sometimes with node modules because GCC doesn't implement node's complicate name resolution scheme completely. To investigate what exactly will break it? Is this a practical concern? Can it be fixed?
-
The page explains that the code must be GCC compliant. So really, all this does is deal with the lack of
goog.provide
. Otherwise it is like the:libs
option of importing real GCC modules. -
To investigate This is apparently "alpha" quality, but I have no idea what happens if something breaks. Compiler error? Subtle error during optimized builds? How do I debug it?
You must keep note of the fact that there are two kinds of dependency analyses going on in a normal ClojureScript project: one performed by the ClojureScript compiler and one performed by lein/boot.
- The ClojureScript compiler doesn't understand library versioning. The compiler doesn't read the version string in the
:dependencies
vector in yourproject.clj
file. The only dependency information passed to the compiler is through (1):libs
and:foreign-libs
compiler options, (2)deps.clj
files inside foreign lib jars, (3):require
expressions in cljs code. None of these directives contain version information. - Note that each of these methods tell the compiler where the relevant source code is on disk. In the case of ClojureScript, the namespace dictates its location on disk. In the case of JavaScript, the location is spelled out in the :foreign-libs directive.
- I think this means that the compiler traverses a dependency graph that begins at the entry point of the program, and uses the dependency information to figure out what to include in the final target.
- I think this means that you can only have one version of a library installed at a time. This seems supported by the fact that Leiningen provides a
:excludes
option that you can tack on to the:dependencies
vector if your dependencies require conflicting versions of one library. You can globally exclude a library and manually include the right version in your dependencies vector. See this stackoverflow answer. - This is different from npm's "maximally flat" version tree, which will automatically install multiple versions of the same dependency if needed.
Example: One Failure and One Success to Use react-dnd in a ClojureScript Project
React-Dnd needs two node modules to work: react-dnd and react-dnd-html5-backend, but I'm only going to try to get react-dnd imported for starters. Both of them come what I believe are Webpack UMD distros:
$ head ReactDnD.js
(function webpackUniversalModuleDefinition(root, factory) {
if(typeof exports === 'object' && typeof module === 'object')
module.exports = factory(require("react"));
else if(typeof define === 'function' && define.amd)
define(["react"], factory);
else if(typeof exports === 'object')
exports["ReactDnD"] = factory(require("react"));
else
root["ReactDnD"] = factory(root["React"]);
})(this, function(__WEBPACK_EXTERNAL_MODULE_2__) {
I'm not an expert on UMD, but my understanding is that they allow you to stick them in a <script>
tag and they will load themselves into a global (basically the last else
clause above). More specifically, here, loading the script in a browser environment will set the ReactDnD
global, just like the yayQuery
example in the ClojureScript documentation above. That seems promising.
I note that the UMD module requires "React". In the global version, it will simply read the "React" global variable and use that. In the AMD and CommonJS variants, it will use the return value of require("react")
.
I add the following compiler options in my project.clj
:
:npm-deps {:react-dnd "2.4.0"}
:install-deps true
And I added (require [react-dnd])
to my core.cljs
.
Doesn't work:
Compiling "target/cljsbuild/public/js/app.js" from ["src/cljs" "src/cljc" "env/dev/cljs"]...
events.js:182
throw er; // Unhandled 'error' event
^
Error: Can't resolve 'react' in '/Users/me/testproject/client-cljs/node_modules/react-dnd/lib'
at onError (/Users/me/testproject/client-cljs/node_modules/enhanced-resolve/lib/Resolver.js:61:15)
at loggingCallbackWrapper (/Users/me/testproject/client-cljs/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)
at runAfter (/Users/me/testproject/client-cljs/node_modules/enhanced-resolve/lib/Resolver.js:158:4)
at innerCallback (/Users/me/testproject/client-cljs/node_modules/enhanced-resolve/lib/Resolver.js:146:3)
at loggingCallbackWrapper (/Users/me/testproject/client-cljs/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)
at next (/Users/me/testproject/client-cljs/node_modules/tapable/lib/Tapable.js:252:11)
at innerCallback (/Users/me/testproject/client-cljs/node_modules/enhanced-resolve/lib/Resolver.js:144:11)
at loggingCallbackWrapper (/Users/me/testproject/client-cljs/node_modules/enhanced-resolve/lib/createInnerCallback.js:31:19)
at next (/Users/me/testproject/client-cljs/node_modules/tapable/lib/Tapable.js:249:35)
at resolver.doResolve.createInnerCallback (/Users/me/testproject/client-cljs/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:44:6)
WARNING: uri? already refers to: cljs.core/uri? being replaced by: cognitect.transit/uri? at line 332 target/cljsbuild/public/js/cognitect/transit.cljs
Jan 16, 2018 3:48:56 PM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: /Users/justinlee/seekeasy/client-cljs/target/cljsbuild/public/js/seekeasy/core.js:9: ERROR - required "react_dnd" namespace never provided
goog.require('react_dnd');
^^^^^^^^^^^^^^^^^^^^^^^^^
Jan 16, 2018 3:48:56 PM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 1 error(s), 0 warning(s)
ERROR: JSC_MISSING_PROVIDE_ERROR. required "react_dnd" namespace never provided at /Users/justinlee/seekeasy/client-cljs/target/cljsbuild/public/js/seekeasy/core.js line 9 : 0
Successfully compiled ["target/cljsbuild/public/js/app.js"] in 31.225 seconds.
Remember how the documentation said node modules won't always work?
The Node.js module specification varies slightly from the CommonJS specification in that the module identifier that is passed to require() doesn’t always need to be an absolute or relative path. This makes it difficult for the Google Closure compiler to resolve the dependencies of a node module since th compiler was implemented following the standard CommonJS specification. Therefore, it might not be possible for a node module to be converted to a Google Closure module.
I think this is what it was talking about. That require("react")
in the UMD header is possibly what's causing this problem.
Some question and points about the above:
- Everything from "SEVERE" on down gets eaten by figwheel, so you need to run
lein cljsbuild once
to see it. - If you run
lein cljsbuild once
again, it doesn't do anything! - Why in the hell did compilation succeed?
- Why is the compiler trying to load the UMD dependencies during compilation? If it is going to do this, shouldn't it be reading the
package.json
? This must be the difference between CommmonJS and node modules (?). Confusing since this feature is actually callednpm-deps
. - If I'm right, it would be handy to have an option to ignore it since I'm going to provide the react library myself anyway.
Okay I'm out of ideas. On to the next thing.
I've copied the UMD builds into a lib
directory and added this:
:foreign-libs [{:file "resources/lib/ReactDnD.js"
:file-min "lib/ReactDnD.min.js"
:provides ["react-dnd"]
:requries ["react"]}]
Same problem. That's weird. I changed :file
it to point to a nonexistent file. Same problem. Okay the node_modules
are left over. I delete them and the package.json
and try again.
Much better! Good news: I get a sane error message:
Caused by: clojure.lang.ExceptionInfo: No such namespace: react-dnd, could not locate react_dnd.cljs, react_dnd.cljc, or JavaScript source providing "react-dnd" in file src/cljs/seekeasy/core.cljs {:tag :cljs/analysis-error}
Bad news: nothing I do to the :foreign-libs
directive actually does anything. I feel like I have two potential problems:
- I'm wondering if I've got a pathing problem here. Since I get the same error even if I point
:foreign-libs
to a non-existent file, it's hard to say. - I'm wondering if I've specified the
:foreign-libs
correctly inproject.clj
. Hard to say!
Turns out to be neither. The template I'm using has two builds configured and lein cljsbuild once
either builds both or only builds the :min
build. I had only added the :foreign-libs
diretive to one of the two. For the record, you get a good error message if you specify a nonexistent file in :foreign-libs
. Whew.
I still get a runtime console error:
base.js:1357 Uncaught Error: Undefined nameToPath for react
Per this stackoverflow post, I change it to cljsjs/react
.
That's it! I can access js/ReactDnD
now.
One other point: completely eliminating the :requires
key also works. This confirms my understanding that the compiler require-tree is only needed to ensure everything ends up in the final bundle. Since I'm including rum
and rum
includes react
, its presense in the :foreign-libs
entry for react-dnd
was technically redundant (though obviously I'll keep it in).