Do you spend countless hours searching through code repositories or file systems? Do you want to use slow and outdated search tools like grep? Look no further than the Silver Searcher, the lightning-fast file search utility designed to revolutionize how you search for files. So say goodbye to frustratingly slow search times and hello to the power of the Silver Searcher.
The Silver Searcher, also known as Ag, is a fast, powerful, and easy-to-use file search utility for CLI. The Silver Searcher is particularly useful for searching through large code repositories, as it can quickly search through thousands of files with regular expressions.
Here are some key features of the Silver Searcher:
- Fast searching: The Silver Searcher is optimized for speed, using multiple threads and fast I/O to search through files in parallel. This makes it much faster than traditional search tools like grep, especially when searching through large code repositories.
- Powerful control over what files you search: respects .gitignore, allows regex for filenames, and has many filetype definitions.
- Regular expression support: The Silver Searcher supports regular expressions, allowing you to perform complex searches with ease.
- Smart case sensitivity: By default, the Silver Searcher is case-insensitive when searching through files, but becomes case-sensitive when you use uppercase characters in your search query.
Simple Usage Patterns
To use the Silver Searcher, simply open a terminal window and type “ag” followed by the search term you want to find. For example, to search for the term “hello world” in all files in the current directory and its subdirectories, you would type:
ag "hello world"
By default, the Silver Searcher will search through all text files in the current directory and its subdirectories. You can also specify specific files to search using the “–file-search-regex” option. For example, to search only through files with the “.txt” extension, you would use the following syntax. Also note ag has predefined filetypes like ‘–shell’. You can get a list of them with ‘ag –list-file-types’.
ag --file-search-regex "\.txt$" "hello world"
Speed Showdown
I used recursive grep with large code bases for years. Unfortunately, it can seem really slow. That’s why I was so pleased when I found ag! It’s hard to see a significant difference in performance between grep and ag with a small set of files, so let’s clone a copy of the Linux kernel to use as our test bed. We’ll run each command several times to ensure each is leveraging the filesystem cache as much as possible. I’m running these tests on Mac OS with version 2.2.0 of Ag from Brew and version 2.5.1 of BSD grep from MacOS. I’ll also add version 3.7.0 of GNU grep later for contrast.
➜ linux git:(master) ✗ time grep -r radio .
./Documentation/admin-guide/devices.txt: 176 = /dev/gtrsc Gorgy Timing radio clock
....
./CREDITS:E: HB9JNX@HB9W.CHE.EU (packet radio)
./CREDITS:D: miroSOUND PCM20 radio RDS driver, ACI rewrite
./CREDITS:E: vk2xlz@gonzo.vk2xlz.ampr.org (packet radio)
grep -r radio . 92.79s user 4.95s system 90% cpu 1:48.16 total
➜ linux git:(master) ✗ time ag radio
Documentation/admin-guide/devices.txt
313: 176 = /dev/gtrsc Gorgy Timing radio clock
....
3375:E: HB9JNX@HB9W.CHE.EU (packet radio)
3529:D: miroSOUND PCM20 radio RDS driver, ACI rewrite
3570:E: vk2xlz@gonzo.vk2xlz.ampr.org (packet radio)
ag radio 3.01s user 5.30s system 285% cpu 2.910 total
You can see that ag is remarkably quicker than grep: 2.9s instead of 1 minute and 48 seconds. What’s also apparent is ag is leveraging multiple cores as the system utilization is 285%. Anything over 100% means it used more than one core. Also of note is that ag is doing a case-insensitive search instead of a case-sensitive one, which is more demanding. You can, of course, do a case-insensitive search in grep or change ag to do a case-sensitive search. I like ag’s case-sensitivity model better as it fits my usage more often.
Can we make grep faster? Yes. Proponents of grep will argue that modern versions of grep are much faster, and even faster than ag in some cases. Indeed, modern versions of grep do use multithreading, for example. In addition, you can string together combinations of GNU find and grep to operate on a large code base much more efficiently. However, it can be difficult to craft the proper command and in my experience grep is still much slower. Let’s try again using GNU grep (ggrep) on my Mac. I installed ggrep using Homebrew.
➜ linux git:(master) ✗ time ggrep -ri radio
Documentation/admin-guide/devices.txt: 176 = /dev/gtrsc Gorgy Timing radio clock
....
CREDITS:E: HB9JNX@HB9W.CHE.EU (packet radio)
CREDITS:D: miroSOUND PCM20 radio RDS driver, ACI rewrite
CREDITS:E: vk2xlz@gonzo.vk2xlz.ampr.org (packet radio)
ggrep -ri radio 2.61s user 3.08s system 39% cpu 14.400 total
You can see that the newer version of grep is much faster than the BSD grep that comes with MacOS 11. I even switched to using a case-insensitive search to compare it more uniformly to ag. However, ggrep is still almost five times slower than ag.
What if we try and optimize grep’s performance by skipping over files we don’t want to search? I listed a few commands and their respective times in the table below. GNU grep seemed quicker than ag at first. However, the command only appears faster. It’s producing invalid results because it’s only searching .h files. The BSD grep was also getting a big pass. After debugging things, I found find isn’t running grep/ggrep against any of the .c files. Though find produces the correct file-list when not using the -exec switch, it fails to send the entire list to the -exec command. This could throw a horrible wrench in your dev session! It can be difficult and tedious to get large commands like that correct.
| Command | Time |
| ag –cc radio | 2.755s |
| find . -name “*.c” -o -name “*.h” -exec grep -i radio {} + | 12.078s *bug |
| find . -name “*.c” -o -name “*.h” -exec grep -i radio {} + | 1.549s *bug |
Not only is ag much easier to use, but it always produced correct results during my testing in this article. Here’s a list of other possible ways to use ag in your daily work.
| Command | Description |
| ag –cpp “#include” | Find #include lines in .cpp .cc .C .cxx .m .hpp .hh .h .H .hxx .tpp files |
| ag project –file-search-regex CMakeLists.txt | Search all CMakeLists.txt files for ‘project’ (case-insensitive) |
| ag PROJECT –file-search-regex CMakeLists.txt | Same command as above, but only return lines that match uppercase ‘PROJECT’ |
| ag Project –file-search-regex CMakeLists.txt | See two commands above, but only match ‘Project’ with case-sensitivity |
| ag -s project –file-search-regex CMakeLists.txt | Search all CMakeLists.txt files for all-lowercase ‘project’ (case-sensitive) |
| ag -a foo | Search all files, including those listed in paths in .gitignore |
What Did We Find?
Overall, the Silver Searcher is a powerful and flexible search tool that can save developers and sysadmins a lot of time and frustration when searching large code repositories. In addition, its speed and ease of use make it an indispensable tool for anyone who needs to quickly and accurately search through large amounts of text-based data. Use the link at the top of this post to find more info and install and try it today!