Showing posts with label open source. Show all posts
Showing posts with label open source. Show all posts

Sunday, February 21, 2016

My first HBase project milestone reached: a new HBase-oriented Maven archetype infrastructure, authored by me, is now part of the core product

Image courtesy of NASA
My first adventure in contributing to the big-data-oriented open-source software world has reached its first major milestone: a new infrastructure for development and management of HBase-oriented Maven archetypes (authored by me) has now been pushed through to the main branch of the Apache HBase project (which is to say, it is now a formal part of HBase). While my offering is not yet publicly available (that still requires a backport to the next 1.x release of the product), it is formally "in the can", and I'm very pleased!

Maven archetypes, to me, represent some of the most useful tools in state-of-the-art Java development. Generically speaking, they are used to allow very quick and robust automatic generation of a complete development environment (in Maven project format) relating to a specific technology (e.g., Hadoop, MySQL, etc.), often including fully-functional example code, so that a developer can get right to work understanding and using the technology, while spending close to zero time in the often-frustrating task of getting a workspace within their IDE properly configured. All of that is automatically and instantaneously done for them by the archetype.

When I saw the request go out for development of HBase-oriented archetypes (as an entry in the HBase project's JIRA system), I claimed it almost immediately. The fact that it was posted by one of the most prominent members of the project's management committee suggested to me that it was no idle wish-list item, but a serious request for development. I knew absolutely nothing at that point about Maven archetypes, nor how to develop them; but neither, apparently, did anybody else among all current HBase contributors. It would be my chance to study up on a new-to-me technology and apply it in a completely new-to-the-world (but very-useful-to-the-world) way. In other words -- it was a chance to do yet again what I've successfully done so many times before in my software engineering career with other technologies, but this time in an open-source context!

Not long after beginning to acclimate myself to the Maven archetype world, I realized that if I simply created the archetypes being requested and "dropped them into" the HBase project, that in all future builds of HBase there would be no proper testing of significant components of the archetypes. Just as problematically, it would leave things difficult to manage in the future for those who would need to (1) maintain the existing archetypes and (2) create new archetypes for different purposes. The available tools for producing Maven archetypes did not seem to lend themselves to the full software development lifecycle, and they are not particularly well-documented in all necessary regards. So my additional goal became to provide a very simple and well-documented infrastructure for development, maintenance, and future additions to a collection of HBase-oriented Maven archetypes.

In the end, I have provided an infrastructure which (1) provides for ongoing testing of all components of each archetype as new HBase builds are done, (2) allows future HBase contributors to add new archetypes in a straightforward away, via brief documentation provided in the README.md text found in the hbase-archetypes subproject's root directory.

Now we proceed on with backporting HBase-oriented archetypes to an upcoming 1.x minor release (likely 1.3), and then deploying the archetypes to the Central Maven Repository, allowing for public access to them.

Besides this, I have a few other projects on my plate, all in the open-source realm, consisting of:
  • a new Java-collection-oriented tool-set that I am currently finalizing, which I plan to ultimately offer up to the Apache Commons project, and
  • another HBase-oriented tool-set (unrelated to archetypes) that I'll be publishing on my own and then offering up to the HBase project as a candidate for adoption into the main product.
Other than that, as Mr. Bennet said, I am quite at my leisure, and will entertain any serious proposals (or even idle-but-intriguing proposals) for future software engineering engagements. Please feel free to contact me at:

daniel [at] commonvox [dot] org

Thursday, May 14, 2015

Two new open source Java projects launched on GitHub & Sourceforge

In the world of Software Engineering, I've heard it said that "code is the new resume". 

If that's so, then I've just burnished up two fresh software engineering "CVs" and posted them to GitHub and SourceForge.

IndexedCollection: A Java class library which extends the standard Java collections framework to provide the IndexedCollection class, which offers simple, automatic, in-memory NoSQL composite-indexing of a standard Collection of objects. In many situations it could be a nice alternative (or a simpler complement) to complex ORM implementations, making for higher efficiency and lower TCO in the development and maintenance of Java applications. 
Github: https://github.com/dvimont/IndexedCollection 
SourceForge: https://sourceforge.net/projects/indexedcollection/ 
Gist (simple usage example code): https://gist.github.com/dvimont/4f0e11c79efb775bc893 
Side note -- I've been trying to benchmark my IndexedCollection against another package called CQEngine, but unfortunately that other package's documentation is so sparse, I can't even figure out how to use it for the simplest of queries. The sample code that is published for it won't run against the current release, and both the explanatory text on Google Code and its Javadocs documentation are also out-of-sync with the actual code! So, from a usability perspective, the project seems to have been abandoned, even though the package itself was updated as recently as early 2014. Oh well...

LibriVox Explorer: From a technical point of view, this Windows/Mac/Linux desktop application makes extensive use of the above-mentioned IndexedCollection Java class library, providing the best currently-available example of how IndexedCollections are intended to work. From the end-user viewpoint, LibriVox Explorer is a new way to experience the LibriVox collection of Public Domain audiobooks, and it takes full advantage of the "eye candy" potential of that collection's vast body of intriguing cover-art.
Download it now, and take it for a spin on your desktop:
http://commonvox.org/le-download 
Github: https://github.com/dvimont/LibriVoxExplorer 
SourceForge: https://sourceforge.net/projects/librivoxexplorer/