We've been using
Scrapyd is a service for running Scrapy spiders.
It allows you to deploy your Scrapy projects and control their spiders
using a HTTP JSON API.
HTTP server which provides API for scheduling Scrapy spiders and making requests with spiders.
They don't have thaaat much in common. As you have already seen you have to deploy your spiders to scrapyd and then schedule crawls. scrapyd is a standalone service running on a server where you can deploy and run every project/spider you like.
With ScrapyRT you choose one of your projects and you
cd to that directory. Then you run e.g.
scrapyrt and you start crawls for spiders on that project through a simple (and very similar to scrapyd's) REST API. Then you get crawled items back as part of the JSON response.
It's a very nice idea and it looks fast, lean and well defined. Scrapyd on the other hand is more mature and more generic.
Here are some key differences:
urlargument which as far as I can tell overrides any
I would say that ScrapyRT and Scrapyd very cleverly don't overlap at this point in time. Of course you never know what future holds.