 |
DESCRIPTION
Blue Crab is a versatile program that you use to copy the contents of a web site to your computer, in whole or in part.
- Conduct fast offline browsing and searching without an internet connection.
- Perform batch downloads of URL's
- Create a snapshot of a website for historical archiving.
- Collect specific types of resources such as images or email addresses.
- Search current content more thoroughly than a search engine right on your own computer.
- Check a site for broken links, or generate an HTML sitemap
- Create full page images of URL's (JPEG, BMP, TIFF, PNG, PSD, etc.)
With Blue Crab you can download all the content including HTML, PDF, graphics, video, file archives, etc., or use selective filtering to restrict downloads to specific kinds of files. For example, you can choose to save only the JPEG images Blue Crab finds, or just the PDF's.
Blue Crab has a special feature called the "Media Grabber" which you can use to easily download just the images (or movies) on a web site Moreover, you can view a slide show of the images as they are downloaded. You also have the option of "flattening" the images, i.e. putting them all into one folder, or preserving the folder structure on the server (just as when downloading a complete web site for offline viewing.)
Blue Crab is always gentle on the server, processing only one URL at a time. (You can even make it pause between successive downloads so as to not overwhelm the server.)
The demo version you download now is fully functionaly and runs for about ten days.
Major Features:
- Unique URL extraction algorithms yield very complete downloads compared to other products of this type.
- You can download web sites either for offline browsing, via re-linking of URL's, or to create virtual copies of them on your hard drive, i.e. backup a web site as-is.
- Supports both HTTP, HTTPS protocols as well as server realms. Detailed Grabber window shows server HTTP header response fields.
- Optionally display web pages as they are crawled using either your own web browser or Blue Crabs own web window.
- You can restrict a crawl to just the initial domain, or allow the program to "stray" into offsite URL's
- Blue Crab can remap "dynamic" URL's which contain path and search arguments so that resulting disk files are navigable offline.
- Image URL's into JPEG files (or any other graphic file type supported by Quicktime) to create complete pictures of web pages not possible simply by taking a single screen shot yourself.
- Blue Crab provides its URL imaging ability as a service menu item available from other applications.
- URL's can be imaged into files, or onto the clipboard.
- Built-in directory search displays found files as a hierarchical list, with double-clickable entries.
- A "batch window" enables you to download your own list of files. Simply drag and drop URL's into the batch window, or import them from an HTML file , or text file.
- Grabbing in "images-only" mode just downloads image files, optionally "flattening" the results (i.e. putting all images into one folder.)
- Reusable configuration settings control what is grabbed, saved or crawled:
- Filter by size, date, file type or even content
- Filter by filename, extension or path
- Bookmarks window for storing frequently crawled web sites
- Supports form submissions to enable you to begin a crawl from a login page.
- Supports cookies for more accurate downloads.
- Generate site maps that consist of a hierarchical display of URL's Such maps can be optionally filtered by extension.
- Find email addresses.
- Customizable user-agent (spoofing) enables you to download web sites which are platform specific.
- Optionally receive email notifications of Blue Crab's progress during a lengthy crawl.
- Dock badging provides visual feedback or progress when the program is in the background.
WHAT'S NEW
Version 4.9.18:
- Improved the speed of the media grabber by avoiding unnecessary downloads (such as resources of other types or to display images for the slideshow.)
- The media grabber slideshow has been extended to work with media types movies and PDFs.
- Download progress of redirected urls is now displayed in the media grabber (and all "quicker grabber" windows in general.)
- Improved the "completeness" of the media grabber by extending the method of determining media of particular types.
Version 4.9.17:
- When using the "Media Grabber" to download images from a web page or web site you now have the option of having them imported automatically to an album in iPhoto called "Blue Crab Import." Select this option on the "Media Grabber" pane of the Preferences window (it is OFF by default.)
REQUIREMENTS
Mac OS X 10.4 or later.
|
SCREENSHOT
|
|
 |
|  |
 | |  |
| Blue Crab User Reviews (13 posts) | Write A Review |
 | Sep 24 2008 |
RC I just downloaded a copy and using it to download a graphics heavy site. The application quits after 5 min or so. It says this is a "fully functional demo" but is this "auto-quit" part of the demo? If so, how can I fully evaluate the program? As I wrote this "review" I already had to restart the program 3 times so that "5 min" interval is getting shorter. I may start looking for an alternative. Too bad HTTrack is not available for the Mac..... iMac G5, 2Ghz, 1GB RAM
(Version 4.9.12) | |
| [ Reply ] | |
 | Sep 1 2008 |
BINARYPACKRAT Yes, this app rocks! It does download images from PHP sites like Blogger. Seems to be very stable as well. The best one of all of the ones I have tried.
(Version 4.9.11) | |
| [ Reply ] | |
| Apr 19 2008 |
ITTY The most important feature I've been wanting for years, from ANY downloader, is the ability to download images from PHP sites. (Coppermine gallery, Gallery II gallery, WordPress gallery, etc.) This is a big feature that should be proudly boasted. Now I can finally download full size images from my favorite artists galleries with lots of options. Tested on MacIntel Leopard 10.5.2
(Version 4.9.6) | |
| [ Reply ] | |
| Dec 10 2007 |
SAME AUTHOR AS HELLOPIG, TRIAL REVIEW i dont know why people say blue crab is slow when operating although downloading can be slower than others. make sure you download the latest version of blue crab. -check whether there is virus. -check whether you have enough memory space and hd space. All these may slow the program down.
(Version 4.9.3) | |
| [ Reply ] | |
| Dec 10 2007 |
SAME AUTHOR AS HELLOPIG, TRIAL REVIEW Site sucker is not really good. it have many limitations. -no javascript, nonlocalize (meaning that you cant download websites with animations...) -cannot download tools from website. Blue crab has many advantages: -able to download flash and 99.9% of all file types. -nice graphic of websites downloaded. -Provide many updates and help -I can guaranteed almost any website can be downloaded. (you have to be patience) Why buy any other software? this product have one of the most updates!
(Version 4.9.3) | |
| [ Reply ] | |
 | Jun 27 2007 |
JPPAGLIARO Some users were complaining of sluggish text field's in the Configuration editor. The cause was: The current browser URL is displayed in certain menu items of the "Grab" menu. As it turns out keyboard processing is related to menu updating, unfortunately, and that caused an overhead when the program was checking for the current URL in the your browser as characters were being typed into certain text fields of the program. This has been fixed by adding an option in Preferences to display full menu items in the "Grab" menu. This option is off by default. To edit the option select "Preferences..." from the Blue Crab menu, and then select the "Behavior" pane. Turn the option on by selecting the "Show full titles in "Grab current web page" menu items.
(Version 4.9) | |
| [ Reply ] | |
| May 29 2007 |
JPPAGLIARO Well, I don't consider your review "real" for a few reasons, which boil down to a sense of unfairness I have about it. I'll point out some of them: First of all your star rating is highly suspcious. For example, you give stability a 1 star rating yet didn't provide any examples of how the program is unstable. The other ratings also seem a bit harsh. I think that people who give 1 star overall ratings are likely being unfair. Then you list your problems with the program, the first of which was the startup window. Restricting demo programs in such ways is common practice, and I don't think it warrants mention in a "review." When real reviewers want to use the program I usually provide a free password so that the window isn't a usiance and they can conduct their experiements without the window displaying. You also don't point out what specific sites you had trouble with, at least in a personal email to me. If you had then I could at least determine if there is indeed a problem that needs to be addressed or not. I'm just curious about why you won't send me the URL of the websites that are causing you problems. Perhaps I can offer a suggestion on how to get it to work better on those sites, or resolve a possible bug in the program, if one exists. The web is a big place, and I know there are sites that don't work as well as others. But I find it hard to believe that Blue Crab doesn't do as well as others on your sites. And until you can prove me wrong I can't change my opinion on that. But if you can then I will openly admit I was wrong ;-) As for the sluggish text field, I do know that there was once an issue of that nature, fixed a long time ago. I'd like to know what version of Blue Crab you are using to be sure you have that fix, because I am not receiving problem reports from others of that nature. Finally, and most importantly, most real reviews provide the pros and cons of each program, not just the one they are seemingly trying to knock. It seems to me that your "review" is more of a means to another end.
(Version 4.8.9) | |
| [ Reply ] | |
| May 26 2007 |
DINH I'm looking for an application that can download all the files relating to a website for archiving and evaluation. I'd like to relay my initial impressions on this application: * Nag screen very annoying and wastes my time when I'm evaluating which software I want to purchase... * I'm running a newer MBP and with 90% system idle and 1gb of free memory (2GB total)... Typing text into any of the text fields and moving between dialogs is painstakingly slow in Blue Crab. I type.. I wait.. text appears.. I'm not sure what Blue Crab is doing, I've closed and reopened, etc.. No avail.. No other application is behaving that way.. * In creating a new Grab.. I goto view the configuration.. I change some settings. I close the window, it says I can't save the default?! okay fine... Then I say okay.. It doesn't show me where or how to save the settings to another config.. And then it closes it self automatically and looses all the changes I made. Considering that it takes a significant amount of time to type anything into the text fields, and change tabs, this is incredibly annoying. I finally figured out how to goto the configurations section and create a new config. * After creating a grab, which apparently only grabbed 1 file even though I set it to recursively parse the site as would seem intuitive, it seemed like I only had the choice of "Done" and couldn't re-grab and change settings.. I hit done thinking it'd allow me to change settings, but nope, lost everything I typed in. After evaluating the following applications, here is my ranking for website archiving: SiteSucker - The only application to successfully download my test site in entirety. Also the cheapest coincidentally! DeepVacuum - Almost captured the whole site.. But of course missed the backgrounds and other things which I specifically wanted. Second cheapest ($12). Web Devil - Downloaded some of the site. The browsers seem half-baked for site archiving, its more for downloading galleries of images it seems. It is the most expensive application ($35). Blue Crab - Buggy, non-intuitive, and unusable per my previous description. Only file captured after trialing it was a cookies file. Second most expensive app ($25).
(Version 4.8.9) | |
| [ 11 Replies - Reply ] | |
Replies:
| May 29 2007 |
JPPAGLIARO I have a strong suspicion that you didn't set up the program properly, and that's why you had such bad results. In effect you broke it, probably by creating a configuration that led to no download. Most users find the exact opposite, that Blue Crab is far more accurate when downloading websites than any of the other programs you mentioned.
(Version 4.8.9) | |
| May 29 2007 |
Hmm.. Well. So I believe that it would be silly to write an application that is unusably slow. So from that standpoint, I understand that your obviously not experiencing the same problems I have.. And perhaps other people aren't also. My interest in writing the review was simply to offer a perspective on the several applications available for mac users. A CompSci background helps a little too. The tests were done on a MacBook Pro 2.33GHz Intel Core 2 Duo w/ 2GB of ram. (Reasonable at this time I would think) I'm surprised I'm being blamed for breaking the configuration; as I tried to describe, the application would not save any of my changes to the configurations and the only settings I utilized in the test was the default configs. | |
| May 29 2007 |
JPPAGLIARO Correct, I am not witnessing the text field problem you are describing, nor have I received reports from other users of the problem. I test Blue Crab personally on several computers, ranging from PPC to Intel, but I'll look into it anyway. As for the countdown window that displays when the program is in demo mode: it's to remind you that the program is in fully functional demo mode, and to advise you how many days are left in the demo, when it will expire by date, as well as information on how to register the product. As for price, all upgrades are always free, as is technical support. (Also, Blue Crab provides features the MANY other the programs you referenced don't have.) I'd like to take a look at the website you say Blue Crab could not download. Please send me the URL so I can take alook. In most cases users find that Blue Crab is one of the most complete downloaders, often finding pages the others don't. Moreover, it's relinking mechanism on dynamic websites, such as those using PHP, for offline navigation, is much better: you'll find far fewer missing links. On top of that Blue Crab offers many features that your "initial impression" probably missed. Features all the other programs lack. I should send you some "real" reviews of Blue Crab, in published magazines, that do true side by side comparisons, and Blue Crab always comes out on top in features. As for the user interface and behavior. I agree that the user interface is in ways non-intuitive. It's an old program whose interface will be getting an overhaul in due time. But that issue becomes moot as you use the program, and learn the correct way to use it. Finally, I suggest that you take a look at the Blue Crab Tour on my website to help you get started. It's linked in the Help menu of the program. It provides information on using configurations, and other tips. (By the way, the "Version History" is a good way to see the features which have been added over the past year ort so.
(Version 4.8.9) | |
| May 29 2007 |
I'm not sure why your disqualifying my review as not real, is that solely on the basis that yours did not favor well? My review was simple, I downloaded 4 applications that claimed to do similar tasks. I have a website that needed to be archived and I gave each application a few minutes of my time to see which ranked the best in the following categories: I find it very helpful when other people post similar findings and thought I'd do the same. I'm sorry that something about Blue Crab didn't run well when I tested it, perhaps you did everything right and don't have any memory leaks, infinite loops to no where, etc.. All I can report on is my impressions as a user who wants to find the best solution. I appreciate that the upgrades are free and you are fairly responsive to feedback. Down the road I'll have to check back in and see how Blue Crab is doing. | |
| Jun 27 2007 |
JPPAGLIARO Update: The problem with the sluggish text fields that some users were witnessing has been resolved in version 4.9.1.
(Version 4.9) | |
| Dec 1 2007 |
HELLOPIG I feel that blue crab is one of the ost comprehensive website downloader, but there are some disadvantage. 1. takes triple as long to download a website although it is more comprehensive than others. 2 Unable to save settings . 3. If there is an internet breakdown or connection to the internet is broken, you have to start downloading files from point one.Unable to save where you have stop! (the worst point)
(Version 4.9.3) | |
| Dec 1 2007 |
Thanks so much for the feedback. I am working on this problem. I use Blue Crab myself, so I know how annoying it can be to need to start a download from scratch under certain circumstances. However, that said, please note that Blue Crab can save its settings in configuration files, and that one setting in particular can help here. It is called "check file modification dates" on the "Grabber" pane of a configuration. If you sleect that option Blue Crab will only download resources like PDFs, images, movies, etc. if they have changed since the prior downoload. This does not affect the downloading of HTML or relinking, so there is still an overhead to re-starting a download. I hope this helps.
(Version 4.9.3) | |
| Dec 6 2007 |
SAME AUTHOR AS HELLOPIG, TRIAL REVIEW I realized that if there is a break down of the internet, it will report errors and stop downloading. Hopefully you will be able to change that. (stop downloading when there is an internet breakdown/disconnection) Hopefully this comment will improve your product> Wish to get a free copy(registered) of your product!
(Version 4.9.3) | |
| Dec 10 2007 |
SAME AUTHOR AS HELLOPIG, TRIAL REVIEW Site sucker is not really good. it have many limitations. -no javascript, nonlocalize (meaning that you cant download websites with animations...) -cannot download tools from website. Blue crab has many advantages: -able to download flash and 99.9% of all file types. -nice graphic of websites downloaded. -Provide many updates and help -I can guaranteed almost any website can be downloaded. (you have to be patience) Why buy any other software? this product have one of the most updates!
(Version 4.9.3) | |
| Dec 10 2007 |
SAME AUTHOR AS HELLOPIG, TRIAL REVIEW i dont know why people say blue crab is slow when operating although downloading can be slower than others. make sure you download the latest version of blue crab. -check whether there is virus. -check whether you have enough memory space and hd space. All these may slow the program down.
(Version 4.9.3) | |
| Mar 16 2007 |
DRAD You might want to also check out Deep Vacuum although it has confusing options when you get the right combination it works well. What has not worked well for me: Speed Download. SiteSucker used to be great but now not so much because hasn't stayed up with OS X upgrades.
(Version 4.8.3) | |
| [ Reply ] | |
|
| View all 13 posts >> |
|
