Can IJulia Jupyter/Pluto run on Linux without JULIA_COPY_STACKS=1?

[roflmaostc]

Hey,

in the REPL it is possible to run a JavaCall dependent library (like View5D) without JULIA_COPY_STACKS=1.
In IJulia or Pluto it is not:

JavaCall.JavaCallError("Either the environmental variable JULIA_COPY_STACKS must be 1 OR JavaCall must be used on the root Task.")

Stacktrace:
 [1] assertroottask_or_goodenv
   @ ~/.julia/packages/JavaCall/tjlYt/src/jvm.jl:236 [inlined]
 [2] _init(opts::Vector{String})
   @ JavaCall ~/.julia/packages/JavaCall/tjlYt/src/jvm.jl:285
 [3] init()
   @ JavaCall ~/.julia/packages/JavaCall/tjlYt/src/jvm.jl:277
 [4] view5d(data::Matrix{Float64}, viewer::Nothing; gamma::Nothing, mode::DisplayMode, element::Int64, time::Int64, show_phase::Bool, keep_zero::Bool, name::Nothing, title::Nothing, properties::Nothing)
   @ View5D ~/.julia/packages/View5D/cKswI/src/View5D.jl:1023
 [5] view5d (repeats 2 times)
   @ ~/.julia/packages/View5D/cKswI/src/View5D.jl:1021 [inlined]
 [6] top-level scope
   @ In[5]:1
 [7] eval
   @ ./boot.jl:360 [inlined]
 [8] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
   @ Base ./loading.jl:1094

Some libraries, like FFTW.jl, crash when JULIA_COPY_STACKS=1 which is unfortunately a deal breaker for my applications.

Can we resolve this situation?

Thanks,

Felix

[mkitti]

The only other way is for JavaCall to run on the root Task. You can check if you are on the root task by comparing Base.roottask with Base.current_task():

julia> Base.roottask == Base.current_task()
true

If you can first start JavaCall on the root task, then you can use something like https://github.com/mkitti/TaskWorkers.jl to run a server task on the root Task to talk to Java.

Another way to do that is to have two Julia processes. One uses JavaCall while the other runs the notebook. You invoke Java commands by communicating between the two Julia proceses using something like RemoteChannel. At this point though, maybe using something like ZeroMQ via ZMQ.jl makes more sense.

[mkitti]

Looking at IJulia, what you probably would need to do is create a custom kernel.jl which loads JavaCall and TaskWorkers, and then runs @async IJulia.waitloop() just before starting the worker.

https://github.com/JuliaLang/IJulia.jl/blob/cc2a9bf61a2515596b177339f9a3514de8c38573/src/kernel.jl

[roflmaostc]

Hm looks tricky 😕

But would it be possible that View5D could manage the TaskWorkers.jl stuff to ensure that when it is called from within Pluto it calls the root task?

[mkitti]

The key is getting a TaskWorker to run on the root Task before Pluto or Jupyter runs on it. Once you own the root Task, TaskWorkers allows you send arbitrary code to run on it. The underlying issue is that the root Task stack looks normal to the JVM signal handler whereas the stack of other Tasks is nontraditional. COPY_STACKS copies the entire stack including the JVM signal handler.

…

On Sun, May 16, 2021, 12:22 PM Felix Wechsler ***@***.***> wrote: Hm looks tricky 😕 But would it be possible that View5D could manage the TaskWorkers.jl stuff to ensure that when it is called from within Pluto it calls the root task? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#145 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5QOMZLI5DMIWOURXR2WFLTN7WMZANCNFSM444FQRKQ> .

[mkitti]

I'm checking in to see if you made any progress on this.

[roflmaostc]

Sorry, I haven't really tried to solve this.

Maybe @RainerHeintzmann wants to tackle this 😄