Cloje types vs host types, revisited

While preparing the Cloje 0.2 release last week, I came across something I had written, part of the goals for Cloje:

Importing all of Cloje will allow you to write code in a language very similar to Clojure. Or if you prefer, you will be able to pick and choose which pieces of Cloje to import, so that you are still mostly writing code in Scheme/Lisp, but with some added features from Clojure.

I had written that over a month ago, long before I had explored host interop issues. Reading it again during the release preparations made me wonder if I had neglected this use case when considering the host interop API design.

My consideration of host interop was from the angle of someone who was writing in Cloje, and wanted to interact with certain host language features or libraries. I hadn’t really considered someone who was writing in the host language, and wanted to integrate certain Cloje features or libraries. It made me wonder whether it was accurate to say that users would be able to “mostly write code in Scheme/Lisp”, considering that much of Cloje will probably depend on a bunch of new types.

This got me thinking on a more fundamental level about Cloje’s stance regarding host types. Should it be possible (and acceptable/idiomatic) for Cloje users to decide to write Cloje programs that primarily (or exclusively) use host types? Or are Cloje users expected to primarily use Cloje types, and only use host types for host interop?

Note: When I talk about “Cloje types” and “host types” in this post, I am mostly thinking of lists, vectors, hash maps/tables, and strings. Those are the types where Cloje and the host language would have “colliding” types (analogous types with different implementations), and thus the source of possible trouble. Certain other types (such as symbols, numbers, and functions) are the same type in Cloje as on the host, so no conversion (implicit or explicit) would be needed.

To help me organize my thoughts and guide my decision, I have sketched out a few scenarios for different stances Cloje might take regarding host types, and explored some of the implications of each scenario. Continue reading Cloje types vs host types, revisited

Cloje 0.2.0 Released

Cloje icon Yesterday I tagged version 0.2 of Cloje. If you want to play around with it, you can clone the repository. Installation and usage instructions are available in the README.

In terms of features, the biggest changes in Cloje 0.2.0 are the addition of hash sets (an unordered collection of distinct elements), and several set operations based on the clojure.set standard library. Please read the release notes for the full list of changes in Cloje 0.2.0.

Also noteworthy is that the goal of Cloje has changed since 0.1. For reasons described in the Cloje 0.1 Post-Mortem, Cloje will no longer attempt to faithfully emulate Clojure. Instead, the new goal is to bring the best aspects of Clojure to Scheme/Lisp. The first effects of this new direction can be seen in the new set operations, whose behavior is more predictable and consistent than clojure.set when given non-set collections. The old goal would have required me to laboriously recreate Clojure’s quirks and bugs, rather than address and fix them.

Tentative plans for 0.3 are to create placeholder types (vectors, maps, etc.) and an API to convert between Cloje types and host types. This will set the stage for adding Clojure-style immutable persistent data structures in the future. The placeholder types will be immutable, but not (yet) have the structural sharing of Clojure’s types.

Besides writing code, I will also be dedicating a significant portion of my project time to establishing policies, project documentation (how to start contributing, etc.), and outreach to help get more people involved in the project, especially people from underrepresented groups.

Designing Cloje’s host interop API

Lately I have been working on designing Cloje’s host interop API. I had the outlines of a design for the API already, but as I started filling in the details and thinking about different use cases, I realized that my simple design would have had some serious problems in actual use. The host interop API was scheduled for Cloje 0.2, but I have decided to push it back to 0.3 to give me time to get it right.

I wrote this post mostly for my own benefit, to help me focus and organize my thoughts. But, it might be interesting to anyone interested in API design. Continue reading Designing Cloje’s host interop API

Cloje 0.2 Status Update

I previously posted that I was planning Cloje 0.2 to have a variety of new features, and aiming to release it in mid to late June. However, for a number of reasons, I am reducing the scope of version 0.2, and moving the target release date to July 5.

Cloje 0.2 will definitely include the hash-set data structure, and a library of set operations based on clojure.set, as previously written.

I am also working on establishing an API for converting data between Cloje and host-language types. At the moment, Cloje mostly uses the host language’s native data types, so the API will mostly do nothing, but it’s important to have the API available early on. For one thing, it allows users to write forward-compatible code that interoperates with the host language, and will continue to work even if I change how Cloje data structures are implemented. But more importantly, it forces me to think about how host-interop will work, so that potential issues will come to light early in the design process.

Those are the two main features planned for 0.2. It doesn’t feel like much compared to 0.1, to be honest. I considered delaying the release even further so I could put more into it, but that is a perilous and often self-defeating strategy. I think it will be better for the project, and for my own motivation, to have a consistent release schedule, even if that means more modestly sized releases.

Even so, Cloje 0.2 will be an especially small release. This is mainly because I spent the majority of my Cloje time over the past 6 weeks doing research and thinking and planning about open source project management, particularly as it relates to diversity and inclusivity. More writing about that, and announcements of project policy and efforts along those lines, is forthcoming.

Surprises in clojure.set

[Update August 2, 2015: Parts of this post have been revised to “tone down the snark”. This post was a byproduct of frustration with the design of the clojure.set API, but in some places I had expressed that frustration in ways that were merely cathartic and sensational, not constructive or informative.]

Lately I have been implementing the clojure.set API for Cloje, and as usual, Clojure’s APIs are full of surprises, particularly if you pass unexpected types.

Indeed, every clojure.set function I have looked at so far has its own peculiar way of handling this obvious edge case. In general, the lack of type checking results in unexpected or unpredictable behavior that may confuse the user and make bugs harder to diagnose.

Let’s start with some fairly mundane examples, then work our way up to the truly puzzling. I’ll then explain why this is a problem, and describe how I’m addressing it in Cloje. Continue reading Surprises in clojure.set

Implementing Metadata in Cloje

This week I’ve been doing planning for Cloje, and trying to figure out how I might approach implementing certain Clojure features. Even if I’m not going to write the code for several weeks, I want to avoid painting myself into a corner.

One such feature is support for metadata. I see three potential approaches one might take when implementing metadata:

  • Wrap the object with a metadata container
  • Inject metadata into the object
  • Store the metadata separately from the object

Each of these solutions is pretty hairy, and there are significant trade-offs involved. So, I wrote this blog post to explore these approaches and hopefully help me decide which to use. Continue reading Implementing Metadata in Cloje

Announcing Cloje

Cloje icon Over the past couple months, I’ve been tinkering with a new side project: Cloje, a clone of Clojure built atop Scheme/Lisp.

Cloje is partly a fun learning exercise, but it may also be of practical interest to any Schemers/Lispers out there who have wished some of the cool features from Clojure were available in their language of choice.

Cloje currently targets CHICKEN and Racket, two popular implementations of Scheme. One of the chief goals of Cloje is portability. Most of the code is written using standard R5RS and SRFI features, so it would not be very difficult to add support for more Scheme implementations. I will be happy to add Common Lisp as a target, if a Lisper volunteers to help write and maintain it.

Today I tagged version 0.1 of Cloje. This is more of a “milestone” than it is a “release”. If you want to play around with it, you can clone the repository. Installation and usage instructions are available in the README. Future releases of Cloje will be available on each host language’s package manager for easy installation.

Version 0.1 of Cloje includes about 160 functions and macros. Most are reimplementations of basic Clojure features, plus some Java-look-alike functions, and a few additions to fill in some gaps. [Update (May 4): I should mention that Cloje was written from scratch by myself, guided only by Clojure’s public documentation and observable run-time behavior. None of Clojure’s source code was copied, studied, or consulted at any point in the creation of Cloje.]

Tentative plans for Cloje 0.2 include:

  • Syntax for maps, vectors, sets, regexps, and more (optional, pick-and-choose which to enable)
  • Hash-set data structures
  • Lazy sequences
  • Implementing the last few clojure.string functions
  • Various higher-order functions, map-related functions, and memoization
  • Packaging scripts, so Cloje can be made available on each host language’s package manager

Also in Cloje 0.2, I will be changing the primary goal of Cloje. I initially intended to faithfully emulate Clojure’s behavior, with the goal that existing pure Clojure code would run on Cloje. The experience of developing Cloje 0.1 has convinced me that this is not a practical or desirable goal. I’m writing a post with more about this, but for now you can read about the change of goal in Cloje’s goals document. [Update (May 9): The aforementioned post has been published.]

That’s all for now. If you play around with Cloje, let me know what you think!

Atomic Spin Roundup

In mid 2012, I moved to downtown Detroit and joined Atomic Object as a full-time software developer. Ultimately, it wasn’t the right fit for me, and I recently decided to go back to freelancing. Nevertheless, it was a positive experience, and I’ve gained new friends, knowledge, skills, and experience.

Over the course of the two years I was at Atomic, I wrote a wealth of articles for the company blog, Atomic Spin, on a variety of topics related to software development, ranging from specific technologies, to best practices, to community groups and events. Each article was crafted with care and attention. Enjoy!

On Programming Languages and Technologies

On Development and Management Practices

On the Community, Conferences, and Groups