Those of you Linux users out there may be happy to know that version 2.0.5 of HLLib has been completed with full native Linux support. To install it, simply download the source and follow the instructions to build the shared library and executable (HLExtract). Many thanks to Michael Mohr for orchestrating the port.
I mentioned previously that my attempt to add a fragmentation property to HLLib revealed a surprising amount of GCF fragmentation, even after Steam's built in GCF defragmenter had been run. As a result I've released a new version of GCFScape (complete change-log) and HLLib with my own implementation of a GCF defragmenter. As always you can find GCFScape here.
The results of HLLib's defragmenter were better than I had expected with an average speedup on HLLib tests of 2.62 and an average speedup on Steam tests of 2.32 (for a total average speedup of 2.47). The baseline tests were all conducted on Day of Defeat: Source GCF files that had been run through Steam's defragmenter (which claimed 0% fragmentation even though I calculated an average of 8.38%). Read more for detailed test results and instructions on how to use the defragmenter.
Some of you might have noticed a couple problems with my site over the past few days, these problems stemmed from a recent reformat of The WaveLength's server and seem to be all under control at this point. My apologies for any lost comments/registrations.
In unrelated news, I was looking at adding a fragmentation property to HLLib the other day when I noticed something quite odd. It seems the defragmenter built into Steam does little defragmentation at all. In fact, by my calculations, the majority of my GCF files that have seen regular updates are between 2% and 10% fragmented with individual files up to 15%. This is after I run Steam's defragmenter which happily informs me that my GCF files are at 0% fragmentation. (No wonder the thing's so fast.)
These numbers have a huge negative impact on performance. HLLib, which is not optimized for sequential reading, can surpass 50 MB/s (very close to the practical maximum of my hard drive) on unfragmented files, but often performs as slow as 8 MB/s on heavily fragmented files. Practically I would expect a speed up of 1.5 to 2.0 times with proper defragmentation. For these reasons I have been working on adding a defragmentor to HLLib (and subsequently GCFScape).
I've also been hard at work on Crafty, adding real time dynamic lighting for models. This will hopefully lead to more accurate lighting for models as well as normal mapping and other more advanced lighting features.
Last week of summer before school starts, and what better way to lament its passing than by remembering the good times? Read more for some pictures from my trip.
I set out a few weeks ago to squash many of HLLib's inadequacies, and as of today I'm happy to say I've completed my mission. v2.0.0 of HLLib adds the ability to stream and validate files, access abstract attributes, and do so in almost any language with the use of Win32 imports (thanks to a new C wrapper). These features and more have been added to v1.5.0 of GCFScape (complete change-log), which can be found, as always, here.
When I first sat down to write this website I'd never seen PHP before, and like many of my projects to date, I dove head first into it (what better way to learn). Needless to say the design wasn't nearly as good as it could have been. It was full of security holes, made a mess of what standards there were and was a giant pain to maintain.
With v2.0 I've tried to keep what works the same while making it easier for first time users to navigate the site and find what they want. Most old links should be backwards compatible with the new code base. If anyone notices any bugs please let me know. I'm aware that there are still a few oddities with the style sheet, but I'm by no means a master of this stuff.
Hope everyone enjoys the new site design. My apologies for the down time, but it was necessary to update the database.
It's that time of year again. The birds are chirping, the bees are buzzing and the collage students are putting their hard earned education to work back packing around Europe. That's right, as of Friday I'm off on the first (ok mid-afternoon - it's easier to wake up then) flight out of Vancouver directly to London to hang with my sis for a couple of days. After that I'm making my way through the Greek Islands to Turkey and back round to Italy on an epic adventure orchestrated by a good buddy of mine.
Needless to say there aren't going to be any updates around here any time soon. I can't say I'm likely to answer my email either. That's all for now, have a great summer everyone.
I've uploaded an alpha of my Object Viewer (dubbed Crafty) today. This is not a feature complete release, but I've put a lot of work into this thing and I figure it's feature complete enough for most people to find useful. If you're interested in giving it a test run, please read on. You can download the latest release here.
Alpha 7 and Alpha 8 have seen the addition of .obj exportation for most Half-Life 2 formats including .bsp, .mdl and .vmf. This is mostly aimed at the design community which may be interested in high quality renders. Some poor attempts by myself include:
I've done a lot of work on the file system in Alpha 6 to try and get Crafty as similar to Steam as I could. I think I've pretty much accomplished my task, however, it means that Alpha 6 now needs a special gameinfo.txt like configuration for each Steam game. These configurations, called Profiles, automatically mount the necessary system folders and .gcf files to load their respective content and are located under the Setup menu.
One drawback to the Profile approach is there is no way I can include a profile for every modification. If you want to add support for a custom MOD, you must edit the Specifications\GameInfoCustom.csf file in any text editor to add support. It is quite simple, the aforementioned file contains detailed instructions. I will try to add a GUI for this latter.
Just a little update for those of you who are interested, my Source Object Viewer is progressing nicely and will be released as an alpha by the end of this month. Here is a preview of what's in store:
I opened GCFScape up in Visual Studio 2005 the other day just to see how it would look in .NET v2.0. Visual Studio prompted me, asking if I wanted it to backup any files it converted to which I replied "yes". It did an excellent job backing up the various project files, however, it also decided it would be a good idea to alter the source code without telling me or backing it up. Needless to say the project would no longer compile in Visual Studio 2003.
So why am I telling you all of this? Well, instead of trying to manually convert the code back, I decided to go with the flow which is why GCFScape v1.4.0 and up now requires the .NET v2.0 Framework. GCFScape v1.3.1 will be the last version offered with .NET v1.1 Framework support.
If GCFScape v1.3.1 works fine for you, then there is no reason to upgrade, unless you are the sort of person who is interested in Source BSP support and updated widgets, in which case GCFScape has a new home here.
Working hard on my Source object viewer, I added displacement map support for .vmf files today. Not really a huge feat, but it looks cool so I thought I'd post a screen shot summary. Currently the view supports .bsp, .vmf, .mdl, .map, .rmf and .gl files.
Edit: I've added better .vtf support (the program now respects the various embedded flags and mipmaps) along with basic .vmt support (blending and multitexturing) and several new render modes. New screen shots have been included below.
Read more to view the screen shots.
Haven't posted much lately, but that doesn't mean I haven't been busy working on my tools. GCFScape has been updated to version 1.3.1, including a rework of the extraction progress bar and a bug fix for the properties form crash. You can grab it here. VTFLib and VTFEdit have also undergone some bug fixes and improvements, including reworked HDR exposure code. You can grab them here.
I've also been toying with the idea of moving to the recently released .NET Framework v2.0 (as opposed to .NET Framework v1.1) because it fixes several fundamental design flaws that effect some users of my tools. However, the two versions are not compatible, which means I would no longer be able to release v1.1 versions of my tools and users would be forced to upgrade for later versions. (You can have both frameworks installed at once though.) I know how much some people hate having to download the .NET framework, and the new version, while it supports the same operating systems, only supports the latest service packs. Does anyone have any opinions on this matter?
Lastly, I've figured out the light mapping in Source BSPs, which is just one more step towards a Source BSP viewer. It is, surprise, surprise, HDR based. Read more for a screenshot.
I said earlier I would updated VTFEdit with improved HDR capabilities, and today I've made good on my promise adding Tone Mapping to VTFLib. Tone Mapping is the process of taking a HDR image and converting it to a LDR image for regular viewing. This is useful for previewing how a HDR texture will behave in game (see the below picture). More work still needs to be done for HDR texture creation.
As always, you can grab the latest release here.
Recently, Steam was updated with engine enhancements that were added primarily for DOD: Source. Along with these enhancements came the addition of the 7.2 .vtf file format, which is used in a handful of DOD: Source textures. The 7.2 .vtf file format adds support for volumetric textures that, to my knowledge, are only used internally by the engine at this time. The SDK has not been updated.
VTFLib v1.2.0 adds support for the 7.2 .vtf file format and fixes the RGBA16161616F encoder and decoder (this is a HDR format and up until now it wasn't clear how it was being represented). I strongly recommend that users update to the new versions of VTFLib and VTF Plug-In as the previous versions may corrupt 7.2 textures. Note: the volumetric textures have not been tested as I have been unable to get a sample from Valve. So lets just pray it all works... I will try to update VTFEdit with improved HDR texturing capabilities in the future, but right now I am simply too busy.
In honour of the upcoming DOD: Source release, I've updated GCFScape adding a Completion attribute to the Properties form. This attribute can show the download progress of any given folder (root for the whole .gcf file) and is updated in real time as Steam downloads files. Nothing major, but it still provides more information than Steam.
In order to take advantage of this feature you must enable File Mapping and Volatile Access in the Options menu from within GCFScape. As always, you can download the latest version here.
NVidia has recently released an update to their DXT library with several improvements and bug fixes, including the crash happy resizing code. I've updated VTFLib with the new library as well as VTFEdit, improving the WAD conversion tool and adding a batch conversion tool for those of you with large collections to convert. You can download the latest build here; I recommend that anyone who has had problems with the current build and visual quality buffs upgrade ASAP.
Been a while since I had a new post so I thought I'd give it a shot. Most of my tools have received minor updates over the past month, the latest of which being VTFEdit with basic .vmt creation functionality. I've also been working on a Source .bsp viewer. So far I've got solid geometry, materials and PVS\Frustum\backface culling implemented. Major things missing before a release: lightmaps, displacement maps, skybox rendering and static props.
Read more for a screenshot.
I've released a new .vtf utility today, this time it is a Photoshop .vtf file format plug-in. This plug-in will allow you to open and save simple .vtf files through Photoshop using VTFLib. You can download the plug-in and find more information here.
I've uploaded another build of VTFLib today. This release doesn't see too many changes in the library (primarily just a relaxed .vmt parser) but VTFCmd and VTFEdit have both undergone some major improvements. VTFCmd can now create .vmt files in a similar manner to vtex and VTFEdit is awash with new features including a file system browser, toolbar, paste functionality, alpha masking and more. You can download the new version and find screenshots here.
Jed and I have finally got around to releasing v1.0.0 of VTFLib. I'd tell you what it does but you can just read about it and download it here. Note: this is an initial release, the library likely has a bug or two floating around so please, try and break it, then tell us how you did.
Jed of wunderboy.org and I have been hard at work these past few weeks writing a LGPL Steam independent .vtf library for Source Engine developers. The library can read and write Source Engine materials and textures with full support for all known features. It is reaching the end of its development stage and will be released shortly.
I've also been hard at work on a simple graphical frontend for the library. This front end, dubbed VTFEdit, can load and save .vtf files as well as create them from various image sources (.bmp, .dds, .gif, .jpg, .png and .tga). It has mip map, normal map and sphere map generation capabilities and supports both multi-frame textures and environment maps.
Read more for screenshots.
It has been two months since a compromised GameSpy machine killed my site, but today, thanks to the folks at Wavelength, its back up. Two months is a long time, and as some of you may have suspected, it has spawned several updates. So without further ado, here's what's new:
I've released a new version of Batch Compiler with some enhancements to the specification files that are its backbone. These enhancements allow for improved source tool batch generation. Tutorial updates will follow shortly but for now you can view to complete revision history here and download the latest version here.
A new version of GCFScape that can be set to open GCF files opened in Steam (Volatile Access option) has been released along with a shinny new version of HLLib with a vamped up sample application (HLExtract). You can view GCFScape's revision history here and download the latest version here.
Old users should be able to log on without problem and only a handful of posts and other updates were lost due to the migration to the Wavelength. If you notice any invalid links please let me know. Hopefully everything will return to normal now.
During my run though, I noticed that Hammer's compiling interface hasn't improved one bit. As a result I've uploaded some shinny new Source specifications. These specifications were generated by running each tool in verbose mode without any other options. I noticed that some of the options listed didn't work or were misspelled or otherwise lacked a decent description so don't expect these specifications to be perfect.
You can pick the new specifications up on the specifications page, or by running an automatic update from within Batch Compiler. The specification you are looking for is "Source Tools Normal". You can also expect a new Batch Compiler in these coming days with some overdue improvements.
I've released another version of BSP Viewer with many of the features discussed in the BSP Viewer v1.3.0 Thread. Not all of the features discussed have been implemented, but I figured there were more then enough changes to warrant the current release.
Well it's not every day that you reach the 1,000,000 hit milestone (since whenever I started counting) so I thought I'd take a minute to post the news. That's it, time to take a shower.
I've released another version of BSP Viewer which takes advantage of HLLib's capabilities. The new version not only uses HLLib to load .wad files but can open multiple .gcf files to locate and load content from. This means BSP Viewer can now load textures embedded in .gcf files. The new version can also load and display skyboxes.
To take advantage of these new features be sure to set BSP Viewer up properly. To do this:
- Go to the Options menu and select Setup.
- In the MOD field enter the name of the MOD you want to use BSP Viewer for (valve, cstrike, dod, ns, etc.).
- In the UserPath field select your Steam user path (Steam\SteamApps\email@example.com).
- Add all the packages your MOD uses (half-life.gcf should always be added).
You can download BSP Viewer from the downloads page.
Batch Compiler v3.0.0 is finally ready to be taken out of beta. The final build includes several fixes and improvements over the beta builds and as such beta users will want to upgrade. You can pick up the final build here. Special thanks to all users who helped debug the application and its specifications.
I've released a new version of GCFScape that fixes the Failed to map GCF file to memory. error caused by large GCF files. You can download the latest version here.
I've also released a new library called HLLib. HLLib provides an abstract interface for C++ developers that can read the various types of Half-Life packages including GCF, PAK and WAD packages. It is an open source library licensed under the LGPL and includes an example application demonstrating its usage. For more information on HLLib see this page.
Edit: change that to GCFScape v1.2.0 and HLLib v1.0.2. The new version of GCFScape implements all GCF interaction entirely in HLLib. This abstract interface means GCFScape now supports PAK and WAD files as well as GCF files. For more information see the GCFScape v1.1.6 and GCFScape v1.2.0 changelogs.
I've uploaded a shinny new version of Batch Compiler written in C#. The new v3 series maintains the same look and design as the v2 series (in fact the main screen looks the same so I won't even bother posting a screen shot) but it has a much more robust specification management system. This is a beta release so please download it, try to break it, and then tell me how you broke it. I've rewritten and added specifications for all the various mapping programs I know of (HLFix, Map Backup, RESGen, Quake Tools, Zoners Tools, Zoners Tools p15 etc.) so keep an eye open for incorrect parameters as there may be a few floating around.
Users new to mapping are going to want to use the "Zoners Tools Normal" specification and experienced users will want to use the "Zoners Tools Advanced" specification. The main difference between the two specifications is simply the tools that make them up.
You can grab the new version here. If you have any suggestions for Batch Compiler feel free to let me know.
I've uploaded a new version of BSP Viewer with several new features. The new version includes the ability to export BSP files and their textures, load PTS files and more. You can download the new BSP Viewer here and view the revisions here.
It looks like while Steam updated most GCF files to the new format, some GCF files still exist in the old format. Timo was good to point out that the version change has made a versioning number in the GCF file's header quite obvious. GCFScape v1.1.3 Beta adds support for both versions five and six so those of you who were getting errors can now rest easy.
I apologize for the massive amount of recent updates, with any luck this will be the last until Valve decides to change the format again.
Deja vu? A few days ago Valve updated Steam's GCF file format. They removed the data from one section of the GCF file and consequently caused GCFScape to operate incorrectly. Yesterday they did it all again only this time they removed the header from the above section.
I've uploaded another version of GCFScape, this time to address the fact that the June 16, 2004 Steam update caused the current version to operate incorrectly. You can download the new GCFScape here and view the revisions here.
In the latest Steam update (June 16, 2004) the format of the GCF file somewhat changed. The format no longer makes use of the section called the 'Block Entry Usage Map' in the GCF File Format. This section is still present, it just contains an empty header and no data. Because GCFScape makes use of this section and assumes it to be present, GCFScape will exhibit unexpected behavior and frequently crash. I am aware of this and working on a workaround (so you can stop e-mailing me).
Thought I'd post some pictures of a project I worked on in collage called Maximum Thrust (just finished it). Maximum Thrust is a multiplayer hovercraft racing game written in C++ with a OpenGL graphics engine. (None of us were really very good at graphics or modeling.)
BSP Viewer is an application I am working on for a school assignment designed to demonstrate the performance boost the many culling techniques used by BSP files produce. I wrote the application in C++ .NET from scratch this week and thought it might be a useful tool for mappers. BSP Viewer allows the optional rendering of textures, lightmaps, entities, special faces, and edges and is a useful tool for diagnosing what the HL engine is drawing. BSP Viewer does not render skyboxes or models (these features may be added in a latter version).
Users with a small amount of video memory should note that the viewer may lag when you leave the world. This is because when you leave the world the whole world is drawn and it causes a lot of thrashing of your video card's cache. Also, some entities may not be rendered by the viewer. This isn't so much a bug but a problem with how the viewer uses the PVS to determine if an entity is visible and is a pretty rare occurrence. Lastly, be sure to set your Half-Life root directory from the options menu. This is used by the editor to scan for WAD files.
I've made several changes so GCFScape will remain in beta for now, if no problems arise I will re-release it as a mature application. If you find any bugs please let me know here.
Well I've got nothing else to do while I'm waiting for my HD to reformat but bitch about it here:
Last night I noticed that internet explorer was locking up whenever I tried to open a web page. Odd I thought; I haven't even installed anything for at least two weeks so what could be different? I guess I'll just reinstall it then. So I went to Bill's site, I downloaded the IE setup, and I went right to it but as it would turn out you can't reinstall the bloody thing once it is already installed.
An hour or so latter after visiting 95% of all pages on Bill's site I find an article detailing how to 'restore' IE. It advised me to run SFC (System File Checker). Now Bill's definition of 'restoration' must differ from mine because by the time this program was done I had never seen so many error messages popping up in my life. I freaked out and restarted my computer; big mistake.
After being forced into safe mode I was left wondering, just how do I undo what SFC has done? The entire program consisted of a single progress bar, there is no telling what it has changed or what kind of damage it has done. Then it hit me! I'll use System Restore and bang, my problem will be fixed, right? Wrong! I go to launch this wonderful program and it is so bold as to tell me that it requires at least 200 MB of free space to run. I check my hard drive. Well gee that's funny I thought, last time I checked 2 GB was more then 200 MB. No use arguing with Bill though. What with all his money he sure must know his stuff.
Despite all my sincerest of efforts there is nothing I can do to fix it and I'm left to reformat. The moral of the story? Never run any of Bill's handy restoration utilities they only make matters worse.
Ok, I think GCFScape is ready for a beta release so here it goes.
GCFScape is written in a mixture of managed and unmanaged C++ which means it requires the .NET framework. (The reason I chose C++ .NET is because of its low level programming capabilities and its high level form design capabilities. I wrote a whole lot of code to simply figure the format out so I didn't want to spend too much time on the GUI.) I will try to re-release GCFScape using pure unmanaged C++ but it's the holidays and I want a break so it might not be for some time. I will also do up what I know of the GCF file format and release that.
Please note that I do not in any way guarantee the correctness of this program or the content it exports. The program was written by studying large .gcf files in a hex editor so it is entirely possible that the format I observed is incorrect. Although I was unable to figure out how to programmatically scan past what I call the sixth section, I believe the code that scans for the data block header is accurate. The data block header contains five fields (including a checksum) and all of which I know which means it is highly unlikely that the program will accidentally mistake incorrect data for the data block header though it is entirely possible.
With all of that said you can D/L GCFScape from here.
I've been working on a GCF file extraction utility for two or three days now, so far I've figured out about 60% of the .gcf file format (as you can see below) but it's not quite ready to start exporting files yet. A little preview:
It's been a good four months since I last had time to release a build of TG so this one contains a fair amount of fixes and additions. To be honest I can't remember what most of them are but somewhere along the way they added a good 4000+ lines of code so they must do something...
One noticeable fix is that I've finally sorted out the generation of texture coordinates both within TG and in files generated by TG (with the exception of .T3D files). If you're a TG user please D/L the latest version and give it a go. If no bugs are found I'll update it as the final v3.0.0 build once the issue with .T3D texture coordinates is sorted out.
Haven't posted much in the way of news lately so I thought a little explanation was due. I'm back in collage and seeing how it's me last year I've been busy as hell. I'm not to sure when I'll be able to start updating the site (in particular finish TG) but hopefully I'll have a little more time when my next break comes along.
In the mean time I though some of you might be interested in the engine I've been working on for a 3D implementation of the classic game Scorched Earth. The compiler for the engine takes a .tgm file created by TG and splits the terrain up into chunks, each chunk can have an infinite number of texture layers which the compiler blends together from the texturing information in the .tgm file. Here's a somewhat early picture of what it all looks like:
I've uploaded MAP Viewer's source under the GNU General Public License for all to see. It ain't too pretty and it ain't commented but it gets the job done (the main reason for uploading it was to allow people to see how to load .map files). If you make a valuable addition to it let me know and I'll upload it here (giving you full credit of course). The lighting can really use some work...
You can grab the code here.
This is a little program that I've been working on over the past few days designed for fast previewing of .MAP files. The end result if a fast, refined program with several rendering options which will hopefully prove a useful addition to any mapper's inventory.
I was a little bored these past few days so I decided to write a MAP viewing utility. So far so good:
I've just uploaded a three stage tutorial for creating cliffs in TG for Half-Life (though the first two stages can be applied to pretty much anything). This is probably a pretty good read for most users as it covers about 50% of TG's tools and functions. You can find it here.
After over a year of sporadic programming it's finally out. This is a beta release but it seems to be pretty refined, there are a few minor bugs that I know about but overall the program is quite stable and should prove a huge upgrade from v2.3.0.
I'll be releasing it in two weeks top, whether it has all the features I've been planning or not. It's been over a year in development and it's about time I gave it to the public to play around with.
I have completed TGM and MAP IO so after some final GUI work it should definitely be ready for a beta testing. If I can I'll try to squeeze in some Unreal supported exportation types such as .DXF or .T3D but either way, stay tuned, its coming soon.
The below shows TG's .RMF exportation capabilities. Terrain brushes and hint brushes are grouped in separate VIS Groups for easy management and in solid render mode terrain brushes are colored to simulate flat shading in TG.
I've uploaded yet another version of BC. This version only really applies to countries where the decimal place is defined by the comma. Previously BC would force floating point numbers to contain the comma and in turn write these commas to the batch file which gave Zoners a heart attack. Big thanks to Stanislav for pointing this out.
You can pick up the new version here.
Some screenshots of the upcomeing TG 3.0.0:
Read more to view the rest of the pictures.
I've uploaded another utility to the Miscellaneous Tools section, here's an excerpt:
This simple command line program takes a Half-Life WAD3 file and extracts the highest resolution mipmap of each texture as a bitmap. This program can also be used to list all the textures in a WAD3 file.
For more information click here.
It's been a long time since Ive updated this site, but I thought it was about time to give Nems Tools a bit of a face lift and php seemed like a fun language to learn so one week latter here we are.
I've converted about 95% of the old sites content to the new site; however, converting the news posts would have been too much of a hassle so they shall remain gone forever.
The new site is complete with a new commenting system which will allow users to post comments to all articles. I encourage everyone to register as it's fast and easy and requires no e-mail confirmation. And of course, post any comments you may have about the new sites design here.