> it’s better to choose Rust for software engineering and Python for scripting.
Rust and Python work in some scenarios but not all. Unless the native components are all developed already, you will need a Rust programmer on your team. Rust expertise may be available to organizations of a sufficient size and a narrow focus. In my experience, this sort of arrangement requires a team with dedicated resources.
What I encountered more frequently are attempts to use Python as a full stack language. People are not just using it to implement the scripting frontend interface but also trying to implement the backend that does the heavy processing and the numerical computation. As a developer this is a terrible experience. I'm in the middle of replacing for loops in Python with vectorized numpy code but most of my efficiency gains are erased because I still need Python tuples in the end. Yesterday, I had to consider whether exceptions I throw in Cython code can be caught properly in Python.
Research software engineering is one field where you really do need a full stack language. That kind of software engineering requires high performance with limited resources. Julia with some initial patience does deliver a better developer experience in this scenario than the equivalent Python experience for me partly because I do not need to play vectorizarion games, and I can do everything in one integrated environment.
While, yes, interfaces and static tooling could be better, I do think the situation has gotten better over time. There are interface schemes available and additional static tooling is available.
Julia could deliver a better user experience though. Admittedly, the Python tooling to deploy complex solutions has significantly improved lately via tools such as pixi. Julia tools could deliver generic binary code with packages to ease the initial user experience. Advanced users could re-precompile packages to optimize.
The most promising success I have had with Julia is putting notebook interfaces in front of scientists or deploying web apps. My observation in this scenario is that Julia code can be presented as simple enough for a non-professional programmer to modify themselves. Previously, I have only seen that work well with MATLAB and Python. I have not seen this happen with Rust, and I do not expect that to change.
The other observation is that users appreciate the speed of Julia in two ways.
1. Advanced data visualizations are responsive to changes to data. In Python, I would typically need to precompute or prerender these visualizations. With Julia, this can be done dynamically on the fly.
2. Julia user interfaces respond quickly to user input.
While I think Julia has plenty of room to improve, I do think those improvements are possible. I have also greatly appreciated how much Julia has improved in the past five years.
The 0 or 1 based indexing is actually a very superficial debate for people not very familiar with Julia. Note that 1-based indexing is a standard library feature not inherent to the Julia language itself.
The real indexing issue is whether arbitrary-base abstraction is too easily available.
# Correct, Vector is 1-based
function mysum(v::Vector{T}) where {T <: Integer}
s = zero(T)
for i in 1:length(v)
s += v[i]
end
return s
end
#Incorrect, AbstractVector is not necessarily one based
function mysum(v::AbstractVector{T}) where {T <: Integer)
s = zero(T)
for i in 1:length(v)
s += v[i]
end
return s
end
#Correct
function mysum(v::AbstractVector{T}) where {T <: Integer)
s = zero(T)
for e in v
s += e
end
return s
end
Basically, the concrete `Vector` type is 1-based. However, `AbstractVector` is could have an arbitrary first index. OffsetArrays.jl is a non-standard package that provides the ability to create arrays with indexes that can start at an arbitrary point including 0.
The pyproject.toml, package.json, and Cargo.toml are declarative project configuration files. While the Rust community refers to Cargo.toml as a manifest, it is not a comprehensive and detailed list of a build. That is the lock file.
While go.mod does not allow for explicit version ranges, the versions given are the minimum versions. In other words, the versions given are the lower bound of the compatibility range.
Go also strictly follows semantic versioning. Thus the implicit exclusive upper bound is the next major version. This assumes that all minor and patch releases are backwards compatibile and not breaking.
Dependency resolution in Go uses minimum version selection. That means the minimum requirements of all dependencies are evaluated and highest minimums are selected. In principle, this minimum version selection should be time invariant since the oldest versions of the compatible dependencies are used
While the minimum versions specified in go.mod are not necessarily the version of the dependencies used, they can be resolved to the versions used irrespective of time or later versions of dependencies being released.
Other languages do not use minimum version selection. Their package resolution often tries to retrieve the latest compatible dependency. Thus a lock file is needed.
Python packages in particular do not follow semantic versioning. Thus ranges are critical in a pyproject.toml.
In summary, the "manifests" files that the original author describes are configuration files. In some languages, or more accurately their package management schemes, they can also be lock files, true manifests, due to version semantics. If those semantics are absent, then lock files are necessary for compatibility.
That’s very interesting. Most systems I know would pick the highest versions allowed by the ranges. In maven and gradle, for example, at least by default they choose the highest versions allowed. Even if no version range is used, it picks the highest choice even across major versions, which I always thought was completely broken. What does go do if you have two transitive dependency versions whose allowed major is different?
In some sense, Go does not allow you to change the major version. Packages with the same name but different major versions are treated as different packages.
"overengineered" is not the term I would use to describe Python packaging. I would say it is "under-engineered". As in, "Why engineer a configuration file when you can just do it in code?".
This tendency towards what initially seems like the "simple" solution pervades the Python ecosystem and often requires complex engineering to work around later.
>> you can literally write python wrappers of Julia compiled libraries like you would c++ ones.
> Yes, please. What do I google? Why can't julia compile down to a module easily?
That said Julia's original design focused on just-in-time compilation rather than ahead-of-time compilation, so the AOT process is still rough.
> I don't understand why there's so much friction between julia and python. You should be able to trivially throw a numpy array at julia and get a result back.
The experience with this has been quite mixed, creating a new surface for bugs to appear. Used well, it can be very convenient for the reasons you state.
julia> A = collect(1:5)
5-element Vector{Int64}:
1
2
3
4
5
julia> B = OffsetArray(A, -1)
5-element OffsetArray(::Vector{Int64}, 0:4) with eltype Int64 with indices 0:4:
1
2
3
4
5
julia> A[1]
1
julia> B[0]
1
reply