Sign in to follow this  
Followers 0
Gefferz

UAS2 Progress

20 posts in this topic

Well, I know people are working on other things. But UAS is a bit more like my dinosaur-way of programming than this modern .NET/C# stuff people are heading towards. I've been trying to get some understanding of how things hang together, and it's a lot of fun, especially as I can now melee fight things. So I thought I would create one thread with changes and observations about UAS2 and the capture program, and perhaps these might be useful if anyone's still playing with it. I don't feel I've made enough progress to upload a whole new version. Firstly, the capture program has a number of bugs, at least the one I had. It has "vetors" instead of "vectors", and its bad way of building an insert line means that it often creates dud lines. Actually there's a dozen or more other problems with it, with the fields it captures But I've put this to one side. For one thing, it's generating too many records, and this slows down the server launch time. I've had to create a few scripts like ones to remove duplicates. I also created a file splitter on pcap files. The capture program already can deal with a whole folder, so this means instead of reading one file, and often skipping through and missing half the stuff, it deals with the individual thousands of packets, and this generates more data. One problem and this is me understanding things wrong is that you cannot run the capture program without then applying the SQL straight away. If you run two captures, the second one will have duplicate Linker numbers, which will screw up the database. One problem which was annoying me was that the inventory wasn't being saved. Yet there's code there to do it... it turns out the inventory is being saved, but not loaded back. Actually to be precise it's more the announcement back to the client of the inventory, it happens as far as I can see too early. I added a function to delete the inventory (an internal store of objects), and a delayed second calling of create inventory in WorldServer: thus# int DelayedInventory = 0; (at top) [Before switch ( m_bWorldState )] if (DelayedInventory > 0) { DelayedInventory--; if (DelayedInventory <= 0) { // erase the last botched attempt.... m_pcAvatar->DeleteInventory(); cMasterServer::CreateInventory( this ); //Send create packets for inventory contents } } switch ( m_bWorldState ) { case 0: //First login ------------------ and this is set up here if ( pcRecvPacket->GetFlags( ) == 0x400 ) //400 message acknowledged { m_bWorldState = 4; DelayedInventory = 3; cMasterServer::CreateInventory( this ); //Send create packets for inventory contents cMasterServer::SendLoginData( this ); //Send login data (uses CreateInventory information) ---------------- and in avatar void cAvatar::DeleteInventory() { while ( !m_lstInventory.empty( ) ) { SAFEDELETE( m_lstInventory.front( ) ) m_lstInventory.pop_front( ); } return; } defined in avatar.h function prototype void DeleteInventory(); ------------------------- In TreasureGen.cpp there's a line needed for each CreateXXXXX( function) at the end something like cDatabase::AddToInventoryDB(who->m_pcAvatar->GetGUID(), aWand); or whatever the aXXXX is I made spawnitem take a name as a parameter, which makes it easier to spawn things I added a random distance to monster spawning. This was necessary as monsters attack now! MonsterServer needs quite a bit of work, but now a creature sees nearby enemies and goes for them. It also returns to base after. It's more or less right. Combat animation for melee wasn't working at all, and it was driving me nuts. In the end I went back to the earlier UAS1 and found it did work there, so I stole the "combat animation" code, and somehow that worked! But it's hardcoded for one attack. It needs to do low/medium/high. It's possible I mucked this up myself earlier. But I mention this in passing. Here's a question someone might know. I found a good many of the animations (by modifying !animation to default to the last number++ and working through) - the attack ones are 98-106 - but I didn't find ALL of them. In particular I can't find a stabbing animation, just more of a sweeping one. I'll make a list some time. For instance 106 is low kick. And another question: where are the female sounds? Ugh's and Arghs, etc? Looking at the way certain things are loaded - like monster spawns, NPCs, and permanent objects - I think it would be better to either load things up in a non-database way. Or a second program might do the load, and can just sit there in the background, and use shared-memory to supply whatever is needed. Oh and various hard-coded Maximum are too low once you start loading in captured objects. One big problem I can't get to the bottom of is a CLIENT crash that occurs once in a while. When it happens, the client restarted won't then connect to the server, so it's something the server has done to cause the client to crash. I might have to put some kind of a debug in that I can trap on demand in some way. The XP level code caused me to suddenly go from level 1 to level 126! I can't make sense of this line, so I fed in the XP table into a bit of code, it needs to be written properly, but it works now. Here's the line that was there - how was that ever supposed to work? 6 Xor 5? What's that all about? int level = floor( ( pow( (double)(9 * m_dwTotalXP + 6^5),(double)(1 / 5.0) ) - 6) + 1 ); That's all for now. Will post again.

2 people like this

Share this post


Link to post
Share on other sites
Well it's been a week since my last post. I spent a lot of time trying to solve the crashing bug. In the end after messing around with the packet handling code, it seemed to go away. It also didn't seem to happen when I connected a second machine. I've realised a few problems that have created wrong data in the items database. The capture tool is writing to the wrong table for starters, and the fields are not all correct. I almost had some new boots working, imported from a capture file, they showed up in inventory and ground, but could not be equipped because all the flags were wrong. Incidentally, if anyone is playing with the databases as MySQL, I recommend heartily a free tool called Heidi, which is just incredibly better than MySQL toolbox, so easy to show and edit data. I added code to do the right combat animation according to low/medium/high. I also checked to see if you had no weapon, and switched to the unarmed kicks. I added code that reduces stamina, then realised there's code to do this, which is for some reason broken, another thing to look at! I added combat noises in a very limited way. I found code that was executed when a new landblock is entered. To this I added code that made creatures spawn. I got bored just spawning drudge robbers, so worked out creatures which were fully defined (some have bogus database models) and then made a list of them. So running around Dereth now spawns random creatures in your path, and they head for you and attack if they are in range. All in all, it made a kind of playable single player game: edited highlights: [url]https://www.youtube.com/watch?v=i8Y-OoI2oRE[/url] Creatures death animation was not working, so I looked at that too, it almost drove me mad, but eventually it worked. Healing animation was broken too, or worked sometimes. In the end I copied some other animation in, and changed it to heal, and it worked. Player death causes all sorts of state problems that need sorting, so you need to go in and out of combat because the stance is wrong. I wanted to look at the portals.dat file. I found the ACDat explorer tool, or whatever it's called, only works with ToD portal.dat file. I spent a few days and built a new tool so I can see the data, and find if there was some kind of items repository that could be used to generate items. I think there kind of is - I found an area that had model/palette/texture IDs as one lump, but I can't quite find all the necessary information. I'll make this available somewhere, sometime. It works with the old-school portal.dat file, or the new one. So far it only displays and searches ID's. I found some very old tools with helped with this, in VB. They had a bug or two, and seemed very complicated for what is quite a simple file structure. A few brief words about portal.dat for those interested. For each object there is at least a set of files for 3D model, Textures, and Palettes. In the old portal.dat, these are respectively 0x01 files, 0x05 files and 0x04 files. To a large extent, the contents of these files match between old and new portal.dat files, notwithstanding the variations in how the dat files are stored. "Files" with the portal.dats have names which are this ID and an incrementing number, these are tied to an offset in the file, always on a 1K boundary, via a directory. Directories can have subdirectories of up to 61 entries. The Tod Portal.dat file does not seem to have the textures stored in the same way. I think these are stored indirectly, but still internally numbered the same, for the most part. So the 0x05 files are all very short in ToD. This week, once I've corrected the capture tool, or perhaps rewritten in it in part, I hope to be able to add loads of basic armour and other items. It needs some work because at the moment, it fills up with crazy stuff like housing junk, and random corpses!

Share this post


Link to post
Share on other sites
Great work!

Share this post


Link to post
Share on other sites
I was determined to try to work out why clothing and armour is so messed up. Using a modified version of the data capture, I've been tentatively adding bits and pieces of armour and clothing in. But it never seemed to work, the items never equip properly. I went back to first principles, and tried to understand what did work. I tore the code apart where it's doing inventory equip. Then I realised why the few inbuilt startup items of clothing worked - they were linking into data from portal.dat and not from the database vectors, textures and palettes. Looking closely at things, I realised that the index values captured from pCap files were all out of kilter, for some reason they are the wrong index values (and the data itself is wrong). So I realise that I do have a way to do this. And it's through the icon number that's in the capture data. The strategy is, ignore the captured texture, model and palette data. Instead search portal.dat 0x10 files for the icon number, it's an 0x06 number that can be parsed out. When the file is found, the PortalLinker field in items-template can be set - this is enough to make UAS2 load the models, textures and palettes without needing the linked model/texture/palette files. What's more, since the 0x10 files contain multiple definitions for each colour, I can generate an items-templates record for each colour... but since this is a template file, the colour loading can be done later (it's a choice). I've written a portal.dat examiner - it came in handy for this, but I need to make it save off the files now, then those files can be used by the data extractor, so that's what I'm doing next.Then it's back to the data extractor to parse through them to find the icon number. It's all good fun!

Share this post


Link to post
Share on other sites
I can now spawn items of armour and clothing, and wear them! I modified "spawn item" to take a comma, colour value. It needs to know the colour when loading the model. All sorts of stuff needed to make it load and save things properly. There are two nasty bugs in the code that caused the toon to mess up on equipping and unequipping. For the record they are in DatFile.cpp and Avatar.cpp - DatFile.cpp first - the object's palette change count was never being reset. So there's a new line void cPortalDat::LoadItemModel( cObject *pcObject, DWORD dwModelID, DWORD dwColorID, double dblColorValue ) { cItemModels *pcModel = cItemModels::FindModel(( pcObject )->GetItemModelID()); //pcModel - Item model instantiation; used to fetch item variables //dwModelID - 0x00000001 = male; 0x0000004E = female; other //dwColorID - color ID number; determines the CLUT list looked-up //dwColorValue - the particlar CLUT from the clut list if (!pcModel->m_clothingModelLoaded || dwColorID != NULL) { if (dwModelID == 0x02000001 || dwModelID == 0x0200004E) { pcModel->m_bWearPaletteChange = 0; pcModel->m_bWearTextureChange = 0; pcModel->m_bWearModelChange = 0; pcObject->m_bWearPaletteChange = 0; // WAS NOT IN THIS CODE.. every time it loads, it goes up by more and more } And I think this was causing things to be wrong when the avatar first loads - in Avatar.CPP close to the beginning of CreatePacket - the equip flag test should be done AFTER loading the model, or equipping the item will make things go awry. int paletteChange = m_bBasicPaletteChange; int textureChange = m_bBasicTextureChange; //Loop through the palettes and textures of currently equipped items //Check whether they contribute palettes and textures to the character model for ( iterObject_lst itObject = m_lstInventory.begin( ); itObject != m_lstInventory.end( ); ++itObject ) { cItemModels *pcItemInv = cItemModels::FindModel(( *itObject )->GetItemModelID()); if (pcItemInv->m_PortalLinker != 0) { if (this->m_wGender == 0) cPortalDat::LoadItemModel(( *itObject ), 0x0200004E, ( *itObject )->m_intColor); else cPortalDat::LoadItemModel(( *itObject ), 0x02000001, ( *itObject )->m_intColor); } if ( ( *itObject )->m_fEquipped == 2 ) { // by the time you've messed about testing for 0 through // half a dozen indirect pointers, you might as well // have just addded it! // if ( ( *itObject )->m_bWearPaletteChange != 0) paletteChange += ( *itObject )->m_bWearPaletteChange; // if ( pcItemInv->m_bWearTextureChange != 0) textureChange += pcItemInv->m_bWearTextureChange; } }
1 person likes this

Share this post


Link to post
Share on other sites

this stuff is looking really good. Good job.

Share this post


Link to post
Share on other sites

This week I was very happy to find and fix a horrible bug in UAS2.  In case anyone doing the modern emulators needs to know what it is, I will describe. 

Some armor was coming up as corrupted, it basically made the entire avatar black. I traced through a routine that loaded data from portal.dat, "LoadItem".  This routine assumed that each palette entry occurred only once. When it poked in the palette code, using the colour set, it used the index of the set of colours (xIndex) which was 0, then 1 for the first four palette values (offset and lengths) and the last three palette values. This meant it poked in colour codes for indexes 0 and 1 only, and left the remaining 5 zero.  The one that was poked into index 1 is meant for palettes at index 4, 5  and 6. The index needs to be saved and the palette codes stored in a little loop so that the new index keeps match with the one that is used in storing the offsets and lengths. 

There's also another bug, more an oversight, in the coverage code. This meant, when I eventually worked it out, that the order of items in the inventory, that is the order in the items-Instance-Inventory, changed how clothes and armour overlaid each other. After mucking around with all sorts of ideas, I realised that the trick is to sort the inventory first, in a temporary array of objects, in order of coverage.

A second problem with this code is that then the palettes are not right. A higher coverage item might have an overlapping palette range, because say the palette codes for arms are included in the palette codes for coats.  The correct way is a bit fiddly.  (I have code if anyone needs).  You need to make an array of 100 entries, one for each palette slot. This is populated as each item's palette offset and length is dealt with. As the items are now sorted in coverage order, the later items will overlay the palette slots as needed. When all items equipped are done, the array is read through to determine a master palette list, that is of offset and length values, where the array is not zero. 

Armour and underwear now correctly appear, irregardless of inventory order. The method faster because it no longer does a zillion matches against each inventory item to work out coverage. Oh and you can spawn items with colour. Colour is only partially implemented, but I can do SpawnItem NN,Colour. My item grabber code makes a list of what colours are possible (I made a table of each item code, against each colour, and icon). This is complicated!  Generating an item of a particular colour, if that colour exists, makes it change the icon for that item. I had to make the colour get stored against the inventory.  A load more stuff needs to be stored too per item. It should also have a shading value, which is a number between 0 and 1. This works like this - when an item's palette is loaded, it finds a palette file.  This palette file can contain many palettes for that given colour, say if it's 20, the shading value is timed by that count, so a shade of .25 would make it pick colour index 5.

I had some fun putting weight into the game. UAS already had burden in per object. Sometimes it was called weight.  It just meant keeping track of the weight of items picked up and dropped and feeding out the right message.

Making shields and weapons stay wielded when logging out and in was a major headache.  I fairly easily had it restoring that state, but they would not show up in the slots. I created a wielded flag per item... I made a lot of changes to get this to work and I still have to do the same for bows and wands.

I created a primitive loot system. Now you can kill monsters and their corpses include random loot. I revised this a bit, especially as the very first set of loot included a focussing stone -  and made it give more emphasis to junk rather than top notch armour. At this stage, armour and other things are still not randomised - they are just direct copies of the templates. It amused me to make it drop random pyreals, although there's a 100 things wrong with how those are handled (stacking etc). And gems aren't working for me, they don't appear at all.

So now you can hunt, find weapons and armour and equip those items, and they all appear and work, sort of.  Except all melee attacks are currently reported as bludgeoning!

Doing all this made me realise I've a lot wrong with the items as they've been acquired and due to a fault in my own capture program, I'm missing many types of things, jewellery for instance. On the other hand I've changed the items grabbing code to check for duplicates properly.. so I've not filled the database with duplicate stuff -  but more work to do there, and that's what I'm doing next. Then it's back to the monsters to work out why only some of them work.

 

 

1 person likes this

Share this post


Link to post
Share on other sites

Looking at the monsters tables today. Monsters_type defines critters that can be spawned. It had the wrong linker value, was the ID of monsters_templates records, not dwLinker. Knocked up some quick C++ to fix, and zeroed records that weren't there. Now there are 100's of creatures that can be spawned, hurray.

 

2 people like this

Share this post


Link to post
Share on other sites

This sounds great. 

Hopefully at some point you can release your work and allow people to play the game pre tod. 

1 person likes this

Share this post


Link to post
Share on other sites

Now adding monsters into my data capture program. It's hugely faster than the old buggy capture program. Runs against the large capture files that were previously taking hours, and takes a few minutes to do a whole folderfull.  But it's only doing items or monsters... so that's partly why.

Found it was adding NPCs, but can be eliminated by ignoring records with dwUnknown_V2 = 32. Was pleased to see my first Tusker Guard, and mite scion.

Fixed nasty bug in generating bogus corpse item that show up with a black square.  It needs to animate bending over when looting, and it's also not updating weight. I added healing kits as a treasure item.  Something horrendous is wrong with level gaining. It tells me that I  need 4 billion experience to get to the next level, at level 12, I guess it's calculating -1 or something as a DWORD, but I've not even looked at that code yet.

I need to find a way to share the Exe and database. But I don't want a cease and desist order on me from WB!

 

2 people like this

Share this post


Link to post
Share on other sites

You are doing awesome work keep it up. maybe github it soon!

Share this post


Link to post
Share on other sites

Here's a video of where I'm at today. A few problems note the empty slot  - some kind of duff item is in the database, or something I've overlooked. Sorry low quality video, capturing video is not working properly with 3D mode on. Nice to see doors working, isn't it? Little things like that are pleasing.

6 people like this

Share this post


Link to post
Share on other sites
1 hour ago, Gefferz said:

Here's a video of where I'm at today. A few problems note the empty slot  - some kind of duff item is in the database, or something I've overlooked. Sorry low quality video, capturing video is not working properly with 3D mode on. Nice to see doors working, isn't it? Little things like that are pleasing.

Looks like it's coming along nicely, impressive work.

2 people like this

Share this post


Link to post
Share on other sites

I liked AC so much more during this time period. It's very nice to see work being done on this still. I do hope you share it!

2 people like this

Share this post


Link to post
Share on other sites
On 4/10/2017 at 0:52 PM, Gefferz said:

Here's a video of where I'm at today. A few problems note the empty slot  - some kind of duff item is in the database, or something I've overlooked. Sorry low quality video, capturing video is not working properly with 3D mode on. Nice to see doors working, isn't it? Little things like that are pleasing.

Wowzers!!! Great work!

I have an old copy of AC on a Windows ME machine. Not sure what the date is on the client or portal.dat. Can't be older than 2005. Would that be any help to you, or anyone? 

Share this post


Link to post
Share on other sites
On 4/10/2017 at 6:52 PM, Gefferz said:

Here's a video of where I'm at today. A few problems note the empty slot  - some kind of duff item is in the database, or something I've overlooked. Sorry low quality video, capturing video is not working properly with 3D mode on. Nice to see doors working, isn't it? Little things like that are pleasing.

Awesome job on the strides you've made with this porject! Please consider posting your version up on github or bitbucket. I know many would appreciate it! I don't have as much coding experience but I'd also love to try and help contribute to the effort. 

Share this post


Link to post
Share on other sites

Thanks for the positive comments. Of course I will share in due course. Hit a bit of a problem in the last week with palettes. It all started when I realised I had no Tumeroks, the capture had rejected them. Looking into it, I found the 3d model for Tumeroks must have changed from the old days. The new model number wasn't in the original portal.dat This led me to ascertain which models were not being used, and by forcing them onto creatures with a similar build, searched for the Tumeroks, and other extinct creatures. Many seemed to be variations on zombies and skeletons, and I found the Tumerok model.  Then when I started looking closely at the values for the palettes, to see why the Tumerok types all looked the same, I realised that the new capture data referenced palettes that didn't exist in the old portal.dat file. I then took a step backwards, and looked closely at the capture data for palettes and realised that the original UAS2 and its companion capturing program were in error, but that a workaround somehow made it work.

The original capture program did a weird thing when it loaded up the palettes, and made a field called Unknown-1 which it stored and presented when it was time to show that creature (or item).  Each palette where it exists consists of a palette file number (in the portal file) and an offset and length, which overlay into the working palette. The values captured looked weird... I realised that the offset, length pairs were in fact actually palette file numbers misplaced in the data, so 6D for length of F was really palette file F6D. Because of the botched way UAS2 used the palette values, these somehow still came out correctly in most cases.  In fact there is no need for an "unknown1" field.  If a palette is present, it has a count of at least 1.  What follows the counts of palette, texture and model is something that looks like a base palette, or a palette for part of the 3d model.  This does not have an offset and length value, but is immediately followed by sets of palette overlays that do. These are often a value, and offset and length of 0. The length of 0 means 256.

I played around for ages with a drudge ravener's palette.  It seems some of the palette values come from that first palette, so it's not entirely that the subsequent palette(s) just overlay.

So I have gone back to the original data, and applied various fixes to the data. We now have at least a correct set of data, and unknown-1 is replaced with BasePalette. I am now writing similar code for items.  Of course I had to go through and put the base palette code into every class that presented palettes rather than the unknown_1 field.

The big digression was trying to match the TOD palettes to the old palettes. This was like the old days for me, doing colour matching algorithms. And I think the results are pretty good.... but haven't helped me get Tumeroks looking correct! It seems that the old Tumerok model does not use the palette at all. The only thing I can do is make them slightly bigger or smaller!

So onwards.. and then I need to get back to what I was doing last week.

 

2 people like this

Share this post


Link to post
Share on other sites

Posted (edited)

Didn't Tumeroks get a rework for ToD? And ToD introduced a second look for them as well? I'll check my notes... can't seem to find anything on Maggie's site. Oh well. That's what I seem to remember ToD did a "graphics upgrade" or something.

Non-Sequitor Miscellaneous Random - found this @commands page on her site though: http://www.thejackcat.com/AC/Odds'nEnds/@command.htm brings back a flood of memories on her site. Sigh.... now back to my regular post, already in progress.

Anyway! Love the work you're doing and the progress made. If we're missing one mob out of all AC offered, I will still play it. Would prefer Morsmen be the mob, not tumies.. oh well. In the end, As long as I can kill undead, zombies, revs, liches, lich lords, dark magus (magi?), I will be fine. :D

Edited by Vroomer
Typo
1 person likes this

Share this post


Link to post
Share on other sites

I cant wait to get my old Grindy AC back lol.

2 people like this

Share this post


Link to post
Share on other sites

Wow! I wasn't expecting UAS2 to be revived and hope that one day, i would be able to play pre-tod AC. Thanks for your work!

1 person likes this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0