# Best way to view huge text files? **URL:** \ **Category:** Factual Questions\ **Created:** [February 14, 2007, 3:36pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205 "2007-02-14T15:36:42Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![ZipperJJ](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/zipperjj/32/211_2.png) [@ZipperJJ](https://boards.straightdope.com/u/ZipperJJ)\ **Post date:** [February 14, 2007, 3:36pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/1 "2007-02-14T15:36:42Z")
I need to view very large text files (SMTP logs) on a regular basis - sometimes up to 2GB in size. Notepad is pretty shaky when you get over a certain size, and flat out refuses to open anything over 1GB. I looked at an app called TextPad which is apparently very good for opening large text files but in the case of the 2GB file I think I might need something that splits the file into smaller pieces that I can open. I’d really like to find something free, if possible. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [February 14, 2007, 3:42pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/2 "2007-02-14T15:42:25Z")
I just use WordPad for anything too big for Notepad. ---
**Author:** ![ZipperJJ](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/zipperjj/32/211_2.png) [@ZipperJJ](https://boards.straightdope.com/u/ZipperJJ)\ **Post date:** [February 14, 2007, 4:11pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/3 "2007-02-14T16:11:01Z")
Nope, still too big. Something as big as 2GB tends to freeze and crash even WordPad. ---
**Author:** ![Mangetout](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/mangetout/32/19_2.png) [@Mangetout](https://boards.straightdope.com/u/Mangetout)\ **Post date:** [February 14, 2007, 4:14pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/4 "2007-02-14T16:14:19Z")
Yikes. I don’t have any suggestions, other than trying a few of the text editors on offer in the freeware section of [snapfiles.com](http://snapfiles.com) and at [freewarehome.com](http://freewarehome.com). Is it possible to get the server to chop the logs smaller? ---
**Author:** ![arseNal](https://avatars.discourse-cdn.com/v4/letter/a/ecae2f/32.png) [@arseNal](https://boards.straightdope.com/u/arseNal)\ **Post date:** [February 14, 2007, 4:16pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/5 "2007-02-14T16:16:37Z")
Textpad is a pretty good free editor but you’re right, there’s no magic way to easily view very large files. So, if you’re comfortable with \*nix-style tools, or the idea of learning how to use some, I recommend cygwin (free). Then it’s a simple matter of typing in “tail -10000 filein.txt \> fileout.txt” to split off the last 10000 (or whatever) lines into a new file. And lots other useful stuff like grep and whatnot. ---
**Author:** ![Fear\_the\_Turtle](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/fear_the_turtle/32/2926_2.png) [@Fear\_the\_Turtle](https://boards.straightdope.com/u/Fear_the_Turtle)\ **Post date:** [February 14, 2007, 4:27pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/6 "2007-02-14T16:27:24Z")
You can try Ultraedit for free for 45 days [http://www.ultraedit.com/index.php?name=UE\_MoreFeatures](http://www.ultraedit.com/index.php?name=UE_MoreFeatures) and see if it does the job. Claims 4 GB file size capable ---
**Author:** ![Noone\_Special](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/noone_special/32/2863_2.png) [@Noone\_Special](https://boards.straightdope.com/u/Noone_Special)\ **Post date:** [February 14, 2007, 4:38pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/7 "2007-02-14T16:38:58Z")
UltraEdit is a good suggestion. Another one in its class (30 day trial) is [EditPlus](http://www.editplus.com) (no claim as to file size – But I haven’t run into any trouble well into the Hundreds of MBs range) One that is completely FREE is [ConText](http://www.context.cx), which claims an “unlimited” file size :dubious: Good luck! 🙂 ---
**Author:** ![Chronos](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/chronos/32/134_2.png) [@Chronos](https://boards.straightdope.com/u/Chronos)\ **Post date:** [February 14, 2007, 9:44pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/8 "2007-02-14T21:44:51Z")
With files that big, I really have to ask what you’re planning on doing with these files. If you want to make some change to every line, according to some pattern, a text editor program isn’t what you want at all (unless you plan to spend a few centuries doing it all by hand): For that, you want a text filter program, like sed or awk (Unix command-line programs; you’ll need to spend some time learning to use them). If you have to make extensive changes, but only in a few places, you have the problem of finding those places, whether by line numbers or by matching some sort of pattern. Almost all text editors can find and scroll to a particular word, but only some give you the option to “go to line 27649”, or the like. ---
**Author:** ![ultrafilter](https://avatars.discourse-cdn.com/v4/letter/u/3d9bf3/32.png) [@ultrafilter](https://boards.straightdope.com/u/ultrafilter)\ **Post date:** [February 14, 2007, 9:52pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/9 "2007-02-14T21:52:14Z")
Unless you’re on a really high-end machine, that file is bigger than your RAM, and probably bigger than your virtual memory as well. Switching editors isn’t going to help. What do you need from it? ---
**Author:** ![ZipperJJ](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/zipperjj/32/211_2.png) [@ZipperJJ](https://boards.straightdope.com/u/ZipperJJ)\ **Post date:** [February 14, 2007, 9:59pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/10 "2007-02-14T21:59:14Z")
Thanks for the suggestions, all. I checked out UltraEdit but it does way more than I needed - I just needed to read the files. In order to read a text file of that magnitude in Windows w/o hogging ALL of one’s resources, one needs to have it chopped into smaller bits. Windows, I’ve found, isn’t very good at opening large text documents or displaying folders with a lot of contents. I found a little app called [Chop](http://www.download.com/Chop/3000-2248_4-10483416.html?tag=lst-0-1) that did what I needed - I told it to cut the 2gb file into 95 30mb files, which I was then able to view one at a time in WordPad. I had looked at others (like TextPad) but Chop was the cheapest ($7.50) one that let you tell it how much memory to use during the process. Sorry, should have been clearer in my OP as to what I needed to do - view or split & view. Thanks, though! ---
**Author:** ![gotpasswords](https://avatars.discourse-cdn.com/v4/letter/g/c57346/32.png) [@gotpasswords](https://boards.straightdope.com/u/gotpasswords)\ **Post date:** [February 14, 2007, 10:44pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/11 "2007-02-14T22:44:06Z")
Is this something that you can use automation for? eg: What are you looking for? There are plenty of log management tools out there that will let you set triggers like “show me all SMTP messages coming to us from xxx domain” or “show me all messages that appear to have spoofed headers” etc… and serve up very manageable reports that would probably be printable on a single sheet of paper. ---
**Author:** ![Musicat](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/musicat/32/20189_2.png) [@Musicat](https://boards.straightdope.com/u/Musicat)\ **Post date:** [February 14, 2007, 10:53pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/12 "2007-02-14T22:53:11Z")
[QUOTE=ultrafilter] Unless you’re on a really high-end machine, that file is bigger than your RAM, and probably bigger than your virtual memory as well. Switching editors isn’t going to help. [/QUOTE] Since when is RAM size plus virtual memory the absolute limitation? Ever heard of paging (thru a hard drive temp file)? Wordstar had that in the 1970’s when 32KB was a lot of RAM and very few files fit into it. I don’t know what the limitations of MS Word are, but can’t it handle files of nearly unlimited sizes? ---
**Author:** ![ultrafilter](https://avatars.discourse-cdn.com/v4/letter/u/3d9bf3/32.png) [@ultrafilter](https://boards.straightdope.com/u/ultrafilter)\ **Post date:** [February 14, 2007, 11:07pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/13 "2007-02-14T23:07:33Z")
[QUOTE=Musicat] Since when is RAM size plus virtual memory the absolute limitation? Ever heard of paging (thru a hard drive temp file)? [/QUOTE] Sorry, I wasn’t clear. A file that’s bigger than RAM + virtual memory can be read, but not quickly. That’s what I was trying to point out: the hardware can only read a multi-GB file so quickly, and switching software won’t affect that. ---
**Author:** ![Revtim](https://sea3.discourse-cdn.com/straightdope/user_avatar/boards.straightdope.com/revtim/32/1042_2.png) [@Revtim](https://boards.straightdope.com/u/Revtim)\ **Post date:** [February 14, 2007, 11:10pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/14 "2007-02-14T23:10:07Z")
Maybe notepad++? It’s free and open source. [http://notepad-plus.sourceforge.net/uk/site.htm](http://notepad-plus.sourceforge.net/uk/site.htm) ---
**Author:** ![Ximenean](https://avatars.discourse-cdn.com/v4/letter/x/aca169/32.png) [@Ximenean](https://boards.straightdope.com/u/Ximenean)\ **Post date:** [February 14, 2007, 11:38pm UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/15 "2007-02-14T23:38:00Z")
[QUOTE=ZipperJJ] Windows, I’ve found, isn’t very good at opening large text documents [/QUOTE] You mean “the Windows applications that I’ve tried aren’t very good at opening large text documents”. Operating systems don’t tend to care how big a file is when they open it. ---
**Author:** ![dwc1970](https://avatars.discourse-cdn.com/v4/letter/d/a183cd/32.png) [@dwc1970](https://boards.straightdope.com/u/dwc1970)\ **Post date:** [February 15, 2007, 12:10am UTC](https://boards.straightdope.com/t/best-way-to-view-huge-text-files/392205/16 "2007-02-15T00:10:08Z")
Just open the Command Prompt (a.k.a. DOS window pre-Windows 95) and enter: **more filename.txt** If you are looking for specific data in the file (I’m sure a 2GB SMTP log isn’t something you just read for leisure), you can use the **find** and **findstr** commands. Enter a **/?** after these commands for info on syntax, usage and options.