Artifactory vs Nexus vs ProGet

Disclaimer: The first version of this article was published in September 2018. Since then a lot has changed. We have thoroughly revisited all three products to update the old article and include these changes.

The days when we could put our release on a shared drive or public FTP and still call ourselves professional software developers are over. We are completely capable of managing our source code by using version control systems in a mature ecosystem of repository managers like Bitbucket or GitHub. But, when it comes to our products, the output of our builds, we are stumped. Binary repository managers (BRM) help with traceability by maintaining a single source of binary truth and can even lower lead times by eliminating redundant builds.

The BRM is where we put our DLLs, JARs, and container images. A BRM can help you manage security and licenses by proxying for public package managers like pip or apt-get.

The experiment

But which one should you choose? And how do you make that decision? We set out to investigate the pros and cons of the most popular binary repository managers. 

In our experiment we chose to compare: 
Artifactory by JFrog (v.7.7.3), Nexus by Sonatype (v.3.26.1) and ProGet by Inedo (v.5.3.10). 

They are all under active development, support multiple repository types, and provide support-services upon acquisition of a license. 

We decided to compare the binary repository managers on the following criteria spanning the entire spectrum from Dev to Ops:

• Repositories
• Flexibility and Searchability
• REST API and Tooling
• CI/CD Promotion
• Cleanup
• User Authentication
• User and Access Management
• Storage
• Extensibility

This gave us the broadest base from which to select a winner. Pricing has been deliberately excluded from the comparison due to the wide variation in how payments are structured according to the functionality on offer. The experiment was conducted in the most common environment - a one-node setup with a pro or basic license. This covers most of the use cases that we see at our customers.

Trial licensing

There’s rarely a one size fits all solution and BRMs are no exception, so you might want to give each of them a spin to see how they fit your way of working. 

You can utilize our install scripts which provide the necessary information to get started quickly with the evaluations.

Artifactory 

Simply register for a trial license on their website and get it in your email. It’s a 30-day trial with possibility of an extension by contacting sales. 

Nexus 

Since the last revision of this article the Nexus team has stepped up their trial key distribution game. Simply visit their website and request a 14-day trial. Shortly thereafter instructions on how to get started will be sent to your inbox. 

ProGet

To acquire a trial license for ProGet visit https://my.inedo.com and in the web UI request a trial license key. The trial evaluation period is 30 days.

Winner 

Artifactory and ProGet wins this category by a small margin as they provide an evaluation period twice as large as Nexus. The speed of acquisition was equal amongst all three contestants and they all provide Docker images for quick evaluation. 

Dev

Repositories

Having support for multiple repository types is crucial if you have various languages on different platforms in your organization. Below is a list of the supported repository types that each platform supports. Use it to compare with your needs.

X= commercial support 
C= community support
U=Unofficial
“()” means the support in last review
“{}” means unofficially supported through other repository types
Green means that the support has been added since last evaluation

For more information, please refer to each vendors own homepage:

• Artifactory types: JFrog Artifactory supports main packages, build tools & CI servers
• Nexus types
• ProGet Feeds
• ProGet Feeds in development

Artifactory

Artifactory comes with a tremendous amount of repositories out of the box. JFrog does not allow user provided repositories.

Nexus

Since the last revision, Nexus has upgraded five community repository types to officially supported types. Another five community repository types have also appeared, massively increasing the number of supported repositories in the last two years.

ProGet

ProGet comes with the least amount of repository types and no option to provide your own.  That said, it’s worth mentioning that the company is engaged in supporting new types upon community requests. See more here.

Winner

After the last review, Artifactory took the crown due to its plethora of supported repository types. Since then, Nexus has clearly stepped up its game and the medal in this category needs to be shared between the two.

Artifactory has a slight lead in the number of supported repo types, but Nexus provides you with OSGi interfaces, enabling you to make custom repository types if needed. 

Flexibility / Search-ability

With all of your binaries in one place the sheer number of artifacts can be daunting, and locating the exact artifact you need can be a challenge. Some organizations have workflows that are not supported by the default repository layouts, so flexibility is a strength.

Artifactory

Artifactory has a built in concept of custom layout that allows you to make your own layouts for different repository types. Every repository has a type (Maven, NPM, etc), and a layout connected to it. Be careful when you use custom layouts because most of the repository technologies used on the client side (like Gradle, PIP etc.) do not have the feature of automatically guessing a non-standard layout. This could have been fixed if Artifactory translated layouts on the fly to the end-point, but unfortunately  that feature is not implemented and it isn't the pipeline. 

When it comes to searchability, Artifactory has many different ways of completing the task. You can search by name, checksum, binary type, properties, and more directly through the UI. If you want to combine all possible ways of searching Artifactory can be queried through its own query language; AQL. This is not available in the UI, but only through their REST API and tools.

Nexus

Nexus provides no layout flexibility, but it does allow you to deploy out of layout. In that way you can only get the artifacts through the REST api, and not through e.g. Maven/Gradle.

Their search features on the UI side are on a par with Artifactory with multiple simultaneously active search criterias, both on name, attributes, version and checksum.

Since 2018 their search API has been introduced, leveraging the same capabilities as the UI. In that way you can search for and download artifacts given several properties to search for. A very big plus for automation here.

ProGet

ProGet does not have any notion of a layout. Instead, you upload the required files to ProGet and it places them in the right hierarchical order according to the repository type. If you can live with this strict way of handling files it’s a clean implementation detail that reduces complexity.

Search functionality is tied to a specific repository feed, forcing you to know where your given artifact is stored before being able to retrieve it. In our trial the search box seemed to function only on the artifact name.

Winner

Artifactory for their own query language, AQL, followed sharply by Nexus. Both Nexus and Artifactory have many ways of searching and the freedom to deploy packages out of standard layout. ProGet seems very narrow, immature in the search function, and provides no flexibility in regards to layout (if you need that).

Interactions REST API + Tools 

A usable web interface is great for human interaction, but in a modern software development pipeline most of the daily interactions will be running in pipelines. While all the test subjects support uploads through tools like Gradle and NPM, operations like repository creation and promotions are not yet standard equipment. Therefore having a great REST API alongside CLI tools to automate your daily work can be of great value to your team.

Artifactory

Artifactory has a thoroughly documented REST API that can be found here. 

It handles almost all possible operations you can think of, from everyday tasks like uploading artifacts, to updates of users and configuring reverse proxies. Their UI also makes heavy use of the API which is a testimony to the maturity of the tool.

Furthermore, they provide their “jFrog CLI”, an application which can access a number of their products, including Artifactory, if you don’t have your REST client handy. Among other things, jFrog CLI supports upload/download/move/copy of artifacts and manipulation of builds. 

For the dotnet fans, Artifactory even has example use cases of putting the tool beside your MSBuild project and using the commands as parts of your build process to upload the resulting artifacts.

Nexus

As stated earlier, since 2018, Nexus has released a new and much improved API. They provide both an Open API browser in their UI under the admin section, as well as Swagger files that document the capabilities of the given version the system is running. 

The Open API spec enables automatic scaffolding of client and server implementations in most popular languages, so writing clients to consume the API is very convenient and provides exhaustive language agnostic documentation of the API. This is a very nice feature.

The API provides endpoints to many (if not all) of the features in Nexus, enabling you to make changes to users, repositories, blob storage, etc. They have clearly taken the time to do this right.

Nexus also uses the idea of a pagination token. Instead of returning a large number of items, it returns a smaller set and the token can be passed with a query to return “the next page”. According to our tests the limit on list assets is 20 items and it cannot be adjusted. The benefit to this strategy is that pagination is trivially implemented if you tool around their API, although the Pagination approach encourages you to be conservative about data transfers. 

To extend the capabilities of the API, Nexus exposes an endpoint to upload and execute groovy scripts. While that should be used with extreme care when running in production, having the ability to aggregate data closer to the source, and only send the summary back, can save a lot of bandwidth. This feature is also explicitly given to roles and users to make sure that it is only available on an absolute “need to have” basis. For use case examples of the scripts endpoint take a look at their Github repository.

Nexus does not officially provide a Nexus CLI, although they have published an attempt to the open source community. Sadly, it seems like there has been no activity for the past seven months. 

ProGet

Proget has a very complicated and sporadically described way of how to upload packages and use the feeds. This means, for instance, that there is no way of uploading a maven package to ProGet through their API. They provide the endpoint and you will have to look up yourself how to use the maven repository, or abstract it away with a 3rd party publisher, e.g. Gradle.

ProGet openly admits to only writing a bare minimum of documentation, and refers to Google searches on how to figure out issues with their API. It goes without saying that this approach is lackluster and should be improved. 

The last time we looked at ProGet’s API it only contained an Assert Directory API and a Package Promotion API. Since the last revision of this article a number of new endpoints have appeared. It now contains Feed Management API, Repackaging API, Connector Health API, and Webhook Management API. These additions make it possible to manage artifacts by means of automation, but it still has some distance to go to completely remove the need for the ui because the API still lacks User Management capabilities.

At the time of writing ProGet has announced that there is a Security Management API coming soon. However, the criticism is still valid until this API is released.

ProGet does not have a CLI tool available.

Winner

In the first revision Artifactory was declared winner in this category, but since then Nexus has improved on each and every point. Their embrace of Open API/Swagger and the ability to extend the API where necessary gives them a slight advantage over Artifactory, but it’s a tight call.

If you are lucky enough to be in the dotnet world ProGet has improved since last time and offers more options for automation. However, there are still complications with uploading Maven and other repository types though the API.

CI/CD Promotion

Artifactory

Artifactory has adopted the notion of builds into its domain model and UI. Combined with the promotion functionality, the domain model supports a rich trace of where your artifacts have been stored previously.The promotion functionality can be accessed either via JFrog CLI, REST, or Jenkins Plugin for Artifactory.

Meta-Data functionality is mature with the ability to store both single-value and multi-value properties on all artifacts and builds.

Nexus

Nexus supports promotion/moving of artifacts. The promotion functionality can only be accessed through the REST API. However, the two repositories that you move between need to be in the same blob-store (look at the backend section for more info) for you to be able to perform a move operation.

Nexus supports tagging of artifacts, although this functionality only seemed to work as intended when using a unique tag per build job. We ran into an issue with moving tagged items to a repository already containing items with that tag. This means that you cannot have multiple unit test stages in your pipeline promoting individual artifacts with the same tag. Sonatype writes in their documentation to use one tag per build job, but this would not allow us to move pieces individually, and all artifacts from a build would have to be uploaded and promoted in bulk. 

ProGet

Meta-data is not supported in ProGet in any way (besides name, description and version). If you want to move/promote your artifact from one repository to another, then you need to do this through the REST api.

ProGet does not provide a trace of where the artifacts have been stored. ProGet does not support tagging of artifacts.

Winner

Artifactory wins due to the traceability of where artifacts have been stored previously. 

They achieve this with the inclusion of a build object in their domain model that makes it easy and concise to move your desired artifacts. 

Cleanup

Retention periods

Having a CI/CD flow means that you are building a lot of versioned binary artifacts. And over a very short period of time you can find your storage device frequently running out of space. Having good control over what is going to be kept and what is going to be removed is essential to keeping the storage at a reasonable level. Some industries even have requirements on what and how long certain artifacts need to be available for compliance. In these cases it is not only essential but paramount to have well-defined cleanup policies.

Artifactory

Artifactory has no build in retention mechanisms. Instead, it relies on its query language, AQL, to identify artifacts and CLI or REST to delete them. Read more.

This makes it a bit harder to do than the others, but much more flexible in creating rulesets based not only on time or usage, but on search queries. If you can make a query, you can delete it.

Nexus

Nexus offers scheduled cleanup tasks for all their repository types by allowing you to specify a filter describing which artifacts the clean up policy is applicable to. The filter utilizes regular expressions for matching files and supports time, downloads, and release type as criterias.

ProGet

Cleanup in ProGet is handled with what they call Retention Rules. These rules allow you to delete artifacts depending on last usage, total number of artifact versions, or in regex style pattern. 

ProGet also has the concept of Quotas. You can either specify if the rules run every time(no quota), when the feed exceeds a certain size, or when the artifacts matching the retention rule exceed a certain size. Once the limit is reached, ProGet will keep the quota at its limit by removing files.

Last time we criticized ProGet for not having a way to test your deleting policy. This criticism is no longer valid as ProGet now has this feature. ProGet also offers a feature called DryRun which allows for test runs without affecting the artifacts.

Winner

Nexus has the least mature ui for creating cleanup rules, but the functionality is on par with ProGet. However, ProGet’s solution is more intuitive which helps when creating policies that potentially delete artifacts not meant for deletion. Despite all that, Artifactory has the most advanced way of cleaning and it’s our choice, even though you need to have a standalone service running to do it. 

Ops

Users and Auth

All the BRMs have their own database where you can create users and they integrate with various other services. Integration to Active Directory or the like is crucial for companies and administrators seeking to avoid having multiple places to add and delete users.

Artifactory

LDAP; Active Directory; comes with support for OAuth on GitHub, Google, Cloud Foundry and OpenID; and supports SAML SSO, so you can configure your own provider like e.g. the password manager LastPass or similar. But, be aware that the GitHub integration only works on GitHub enterprise because it grants login permission to _everyone_ on the instance, meaning everyone with a github.com account can login if you choose that one.

Nexus 

LDAP, Atlassian Crowd, SAML and RUT

ProGet

LDAP, Active Directory and SAML

Winner

Since LDAP is the standard, and the enterprise protocol SAML is also implemented by all, everyone wins. 

Users/Groups and Roles restrictions

Restricting access is a must have feature whether you are a small garage company or a large enterprise. 

All three candidates have the notion of users and groups and roles in one aspect or another. Therefore, we are focusing on usability in our assessment rather than “if” they have restrictions.

Artifactory

has a permission scheme of users, groups, and permissions with emphasis on users that belong to groups that have roles attached. Here, roles are called permissions that describe which repositories different group/users have permission to Manage, Delete/Overwrite, Deploy/Cache, Annotate or Read from a repository.

Nexus

Has one of the most cumbersome and complex user/group/permission systems I have ever seen, and that is not meant as a positive thing at all.

They have roles, but there are privileges on top of the roles. The problem here lies in the amount of different privileges one can add to a role. Adding 100s of different privileges makes it almost impossible for you to navigate what kind of access one might need in order to do a given task.

They do have support for disabling users, i.e. they can’t login.

ProGet 

ProGet has a permission scheme of users, groups, and tasks where tasks are what I would normally call roles. You can create users without assigning tasks and default is “none” i.e. anonymous. There is a small preset group of tasks to define global admin, repo admin, upload and delete, and download possibilities. If they do not fit your needs new ones can be configured.

Winner

ProGet and Artifactory are quite similar, but ProGet has a slight advantage because it has a bit more ability to control other aspects like viewing scheduled tasks and management of credentials. Therefore, the winner is ProGet.

Storage

Artifactory

Artifactory still leads in the number of external file stores supported on the enterprise license, and since last time Jfrog have made the Azure Blob Storage integration available for customers with Pro licenses starting in version 6.15.0. 

However, if you want to take advantage of the plethora of available stores, the enterprise license is still required (costing 10X of a pro license). Artifactory has binary deduplication by hash-value, meaning you are able to have multiple instances of the same file without the filestore becoming bloated.

Nexus

Nexus provides two different backend storage solutions for both for OSS and Pro version: local storage and S3. 

It supports multiple storage spaces called “blob stores” and you can have stores of both types active at the same type. There are some limitations to this, though. A repository can only be in one of the stores at once, and there is no easy way of migrating from one to the other (from a local to an S3 bucket). Another limitation is that you can only move artifacts from repositories that are on the same blob store, so remember to have all repositories for an artifacts life cycle in the same store.

Nexus does not provide any blob/artifact deduplication functionality, a feature requested over three years ago.

In 2019 Nexus released a new storage type called a GroupBlobStore to their professional customers. A group is essentially a collection of blob stores treated as a single blob with two different distribution methods between the blob stores: ‘Round Robin’ and ‘Write to first’ - either alternating between all blobs, or only write to a single store. 

Sonatype suggests that Groups are used for adding more storage to a blob storage space via multiple devices, and/or to spread writes and reads across multiple blob stores.

ProGet

ProGet provides local storage by default and to S3 and Azure Blob by official extensions.

They also offer a 7 step guide for migrating local storage repositories to the cloud (but not the other way around). ProGet does not provide any blob/artifact deduplication functionality either.

Winner

It’s hard to pick a winner here. ProGet is good for what you get out of the box, but Nexus is able to span several blob store vendors and spread a single blob store across multiple devices. You still need the biggest license and setup for Artifactory to really benefit from its capabilities, but at least Azure Blob Storage is now available for Pro users as well. The narrow winner is ProGet. 

Extensibility

Artifactory

Doesn’t have a plugin-marketplace, but it provides a repository with a great number of examples even if the documentation of the individual “user-plugins” is of  highly varying quality and depth. They provide extensibility through “Events”, i.e. “when something happens in Artifactory, do this”, which means you can do nearly everything. However, the catch is that you mostly have to discover how. you find the easiest option of just deleting the plugin-logic and keeping it as a tombstone.

Nexus

Nexus has some plugin-ability and has a marketplace. The interface for the marketplace is not great, but it allows filtering on product and version. There are about 25 plugins that are usable in Nexus.

The installation of plugins is a manual process. The plugins are built on OSGi containers (see).

To install a plugin you need to stop the Nexus service, edit XML configuration files, and download the plugin. In some cases you even need to build the plugin too. This process is painful and should be seen as code injection rather than plugin functionality.

ProGet

ProGet is the only one to offer actual plugin functionality, called extensions. In the UI only a few extensions can be found, called build-in extensions, and installed with a single click and a restart. The full list of available extensions with install instructions can be found here. Currently, 8 extensions can be installed from the UI, with a total of 26 on the webpage. Should the somewhat limited selection not satisfy your requirements, Nexus offers an SDK so you can develop your own. 

Winner

By far the best experience is with ProGet even though the selection of plugins is rather small and have not increased much in size since last time. It just works!

Overall conclusion 

Artifactory supports most repository types and has a good search approach with its own query language, AQL. JFrog provides both a REST API and JFrog CLI to interact with Artifactory, supporting Promotion of different artifacts with full traceability. Cleanup is handled through AQL, providing a very flexible and powerful cleanup mechanism. Authentication can be hooked up to almost every login provider, and provides high customization of access roles and policies. Cloud storage is no longer an enterprise feature. The only downfall is that the extensibility is not as good as in other products like ProGet and Nexus.

Nexus has come a long way with the addition of their new REST API with Open API spec and swagger files. This allows for code scaffolding of both client and server stubs in most popular languages to provide an easy way of consuming and extending the API. The promotion functionality seems a bit immature as there is a strange limitation on allowing artifacts to be moved to a repository with artifacts already containing that tag. On the storage side, Nexus has matured in many ways, both in terms of cloud storage and group stores spanning multiple blob stores.

Coming from the .NET world, ProGet has a stricter way of looking at binary artifacts. If you can live with their more narrow selection of supported repositories, and have a web-centric roll-forward approach to your artifacts, ProGet has you covered. 

Giving a recommendation is not as straightforward as in the previous iteration of this article. The recommendation from us is to look at the different products and their supported repository types before choosing one based on your requirements and budget.

If you have any comments, please do not hesitate to contact us.