http://asawicki.info/ Graphics programming, game programming, C++, games, Windows, Internet and more...
All blog entries, ordered from most recent. Entry count: 1022.
Upgrade to Windows 10 - My Story
I upgraded my system to Windows 10. Free upgrade is avaiable until July 28th for all genuine users of Windows 7, 8 and 8.1, so now it's high time to do it if you don't want to pay for it later. My upgrade went well, but not without problems. Here is my story:
First some basic information:
On my old Toshiba laptop with Windows 7, bought in 2011, the upgrade failed. The system is not broken though - Windows 7 still works. After the failure I checked manufacturer's website and found that there are no drivers for this model for any operating system newer than Windows 7, so it's good to stay this way.
On my new Lenovo laptop with Windows 8.1, bought in 2015, I was able to successfully perform the upgrade suggested by the system. All the devices work correctly. All installed programs and settings are also preserved.
On my PC, with most components bought in 2013, upgrade to Windows 10 also failed. I can remember fighting with this annoying upgrade window and deleting some system files few months ago, so that might be the reason. I was ready to format my system disk and install everything from scratch anyway, so here is what I did:
I could find drivers for Windows 10 for all my components and peripherals and they all work correctly (except only an old, little webcam - ModeCom MC-1.3M, but I don't use it anyway). I could also install all the programs that I need and they seem to work.
I recommend you to also get your free upgrade to Windows 10. I had an opportunity to work with this system a lot and I could say it's not that bad :) I know there are some arguments against the new Windows version, so let's look at them:
There are some advantages of the new Windows as well, especially compared to Windows 8.x. There is no Charms Bar and Hot Corners when you but your mouse cursor in the corner of the screen. Start Menu is back with just few tiles you can configure and the old good list of installed applications. (You can always get even more old-fashioned Start Menu by installing free app: Classic Shell).
But the most important is what's not visible to the naked eye. As a developer I know that a new operating system is not about new looks of buttons and menus or new Calc application, but mostly about new technologies under the hood. Some of them (like Direct3D 12 and WDDM 2.0, to name just these related to graphics) are available in Windows 10 only. Some applications and games will require them to work sooner or later. That's the reason I believe it's worth upgrading to Windows 10 as long as it's free.
I plan to update my blog more often now, so I invite you to come back here from time to time or subscribe to my RSS channel.
Pixel Heaven and Bajtek Special Issue
Do you remember "Bajtek" magazine? I don't, because I was a little kid back then, but older colleagues told me that in 80's and 90's it was a popular Polish magazine about computers (like Atari, Commodore or Amiga - platforms that were in use at that time). Archival issues can be downloaded for free from atarionline.pl.
Now, 20 years after last one, a new issue has been released. It's a single, special issue - Wydanie specjalne: Bajtek. There is my article inside - "Programowanie grafiki dzi¶" ("Graphics Programming Today"). The article describes briefly a history of graphics cards (from first 3D games, through 3Dfx Voodoo and S3 ViRGE, cards from NVIDIA and ATI/AMD, appearance of OpenGL and DirectX, to invention of shaders), shows graphics pipeline of modern GPU-s and mentions the new generation of graphics API-s (Direct3D 12 and Vulkan).
Many people who were interested in graphics programming, games or demoscene at the time of Bajtek magazine, now have a more "serious" job, whether in software development or something completely different, and they no longer have time for this hobby, so they are not up-to-date with advancements in this technology. So I thought they may like a short update on this subject.
The new issue of Bajtek was first shown on Pixel Heaven - a party that took place 3-5 June 2016 in Warsaw. I've been there and I had a great time. There were many different activities, like indie games exhibition, retro gaming zone, lectures and discussion panels.
What software engineering has to do with history, politics or law?
When at school, I always prefered scientific classes (like mathematics) over humanities. Among my most hated classes were: history, geography and language/literature. That's why I chose to become a programmer. But despite computer science is a scientific discipline, I can see that on a higher level, software engineering involves some humanities like, for example, history, politics or law.
History, because in software - just like in real life - you need to know what happened in the past to be able to understand the state of things we have right now. For example, in the field of graphics API-s, someone asked a question on Programmers Stack Exchange: Why do game developers prefer Windows? The best answer is the one that extensively explains how two main API-s - DirectX and OpenGL - evolved over years.
Politics, because top-level decisions are not always made based on purely technical arguments. Going back to graphics API-s, Microsoft decided to push its next-generation low-level Direct3D 12 into Windows 10 only, while Khronos Group defined Vulkan as an open, multiplatform standard. Google was rumored to design its own graphics API, was even asked by John Carmack not to do so, and it finally returned to the negotiation table with Khronos, so Android N will support Vulkan as well. Apple chose different path and did design its own graphics API - Metal. Similarly, in the GPGPU field, OpenCL is a widely supported standard, but NVIDIA succeeded in promoting its own, vendor-specific API: CUDA. HSA is yet another such initiative, led by a foundation. Among its members are: AMD, ARM, Imagination, Qualcomm, Samsung and many others, but the list lacks some big players, like Intel or NVIDIA. So developing software technology is a little bit like doing politics - "Am I strong enough to go against the others or do I need to seek allies?"
And finally, the law. Specifications of programming languages and API-s are somewhat like acts passed by the government. They are written in natual language, but should be as unambiguous as possible, precisely defining each term, specifying what is allowed and what is not. Doing something against the specification is like breaking the law - it may go unnoticed, it may even give you an advantage (like programmers notoriously relying on signed integer overlow in C++, despite formally it's an undefined behavior), but you may also "get caught" (and get a compilation error or invalid results from your program). On the other hand, a compiler or API implementor not complying to the specification is more serious problem - it's like a state official breaking the law against you. You may just accept your fate and go away (equivalent of not using broken feature and looking for some workaround) or you may report it (so the bug will be fixed in new compiler/driver/library version).
So although software engineering is a scientific/technical discipline, I think that on a higher level it can be compared to some degree to humanities like history, politics or law.
RegScript 2 - Parameters Framework
On April 1st I've read a very interesting blog post that doesn't seem like April Fools' joke: Game Development Needs Data Pipeline Middleware. I fully agree with the author. There is a code very much needed in every game engine, game editor and other graphics/music/media applications, written over and over again - the one about storing, editing and serializing data structures of various kinds (basically all objects that make up the state/document/game level of a program), each having a set of parameters of various types (integers, floats, strings, vectors, arrays etc.) - so it makes sense to create a library for it.
I (as probably every game developer) already tried to create such framework many times. Now I decided to share the source of my last attempt: RegScript2 @ GitHub. It is not finished and some design decisions I've made may seem controversial, but at least we could start discussion about the solution. What do you think about it? Feel free to e-mail me at sawickiap__REMOVE__@poczta.onet.pl, comment this post or read and interact with my code on GitHub.
Sztukato 2016 - Festival of Arts and Fashion
18-20 March 2016 in Protokultura club in Gdańsk, Poland, an interesting event took place: Sztukato - festival of arts and fashion (Website, Facebook Event). It involved arts gallery, fair of handmade clothes and accessories, fashion shows, concerts and many other activities. I was doing visualizations during the whole event. It was new and interesting experience for me, as I learned a lot during the event, as well as while preparing for it. I especially gained lots of experience in video editing, as I prepared some prerendered video footage. Depending on the circumstances sometimes I played these videos in a loop, sometimes just showed logos of organizers and sponsors and sometimes launched the abstract/psychedelic visuals generated procedurally by my program.
Here is full gallery of my photos from the festival: SZTUKATO 2016 Festiwal Sztuki i Mody @ Facebook.
I can see many VJ-s use Resolume, but for simple displaying images or videos I used Screen Monkey. It's a free program that I came across when browsing VJ Forums. It has some problems (GUI has some minor bugs and it even stops playing videos sometimes), but it also has many useful features (layers, fade in/out, linking clips in a sequence, Schedule and many more).
The biggest problem I had with Screen Monkey is that it didn't want to play any videos after installation. (My environment was: Windows 7 x64 with latest updates, K-Lite Codec Pack Full in latest version, Screen Monkey version 3.7, video files format: MP4 container + MPEG4 Video (H264) video stream) Solution to this turned out to be:
After going back to my work, I had a thought that there is one big difference between creative work and doing software engineering. When creating something, whether it's an art, writing a book or even coding a small program, you can always come up with SOMETHING even if you lack knowledge, experience or time and the deadline is close. It may be better or worse, client may like it or not, but at least you have SOMETHING and the rest is just a matter of negotiation. When working in software, it's more binary - all-or-nothing. You either meet the specification or not, pass all unit tests or not, you fixed the bug or not. Sure you can also write better or worse code, your solution can be more robust, efficient or better architected, but this has its own problem: Writing bad code increases technical debt, which makes it harder to work with the code in the future while being quite invisible to the client and your manager. On the other hand, when assigned some creative task, you probably launch your editor and start from a blank document every time.
Vulkan 1.0 Released!
Yesterday (2016-02-16) was a big day - Vulkan 1.0 has finally been released. The new 3D graphics and compute API from Khronos Group has a chance to be the solution long awaited in the PC world that will:
Time will tell whether Vulkan becomes popular, common standard. It's not so certain. Microsoft promotes its own Direct3D 12, Apple has its Metal API, NVIDIA develops CUDA, old OpenGL and OpenCL are here to stay. What hardware versions and software platforms will eventually support the new API? What will be the quality and performance of those drivers? Will some good debugging and performance probiling tools become available? Will game developers and game engine developers port their code any time soon? What the reception will be among video/media, CAD/CAM, HPC professionals? I'm very enthusiastic, seeing so many learning materials and code samples available since day one! Just look at #Vulkan and #VulkanAPI hashtags on Twitter.
Some useful links to start with:
How code refactoring can fix stack overflow error?
tl;dr: A very long C++ function with multiple local variables, even if they are not very big and they are placed in separate scopes, can reserve as much as hundreds of kilobytes of stack frame, causing "Stack Overflow" even without bugs like infinite recursion. So you better split your long functions into shorter ones.
Can refactoring (or the lack of thereof) cause application crashes? If we understand refactoring as changes in code layout without changing its logic, we might think that it's just the matter of readability and unreadable code increases chances of introducing bugs. But here is a story in which refactoring actually fixed a bug.
Long time ago in a software project far far away, there was a bug submitted telling that the application crashes with "Stack Overflow" message. It was a Windows app, developed in C++ using Visual Studio. I thought: - I can handle that, it should be easy! Every beginner/intermediate programmer knows about the call stack and surely seen this error at least once when accidentally caused infinite recursion in his code. So my first idea was that infinite recursion happens because of some logical error in the code (that should be easy to fix) or some unfortunate, invalid input data (that should be validated for safety before usage).
As it turned out, this was not the case. After setting up all the test environment and catching the crash in Visual Studio debugger, I looked at Call Stack and noticed that it looks quite normal. Sure the call depth was significant (as for C++, I'm not talking about Java here ;) and there was even some recursion, but 20 or 30 functions is not that much. The stack ended with a call to non-recursive function that seemed correct, so it was not the recursion that caused stack overflow.
My second idea was that some of these functions allocate some big objects (like arrays) by value, as local variables on the stack and this causes the stack to grow too big. I reviewed code of the functions that I found on the stack and used "Immediate Window" panel to quickly check sizeof(xxx) of variables or their types when they used some class, but I didn't find anything particularly big. Local variable sizes varied from few bytes to at most several hundred bytes and I couldn't find any big arrays defined in these functions. I also fetched address of some local variable in a function near the bottom of the stack (which looks like 0x000000000009a370), address of a parameter from the function at the top of the stack and subtracted them to see how big the stack grown over all these calls. The result was around 50 KB - not that much.
My third idea was to check maximum size of the stack. It is 1 MB by default, but it can be changed in Visual Studio project settings, in Linker > System tab, as "Stack Reserve Size" parameter. I check my project and I found this parameter not changed from its default value.
OK, now this became more difficult than I thought. After many debugging sessions, where I looked at various pointers, addresses and numbers trying to spot some memory override, stack corruption, out-of-bounds indexing etc., I finally opened "Disassembly" and "Registers" panels. I'm not a fan of such low level stuff, so it took me some time and few Google queries to understand these RSP, RBP registers and make sense of some x86-64 opcodes. While debugging step-by-step in the assembly, I found something interesting. At the beginning of my function, there was a call to mysterious function
__chkstk and the crash occurred inside it. That was a clue I could use to ask Google what this all means. I found this: Description of the stack checking for Windows NT-based applications and this: What is the purpose of the _chkstk() function? These articles say that as the stack grows, next 4 KB pages are reserved. Each next page is allocated by the system on first "touch". I could actually see in my debugger that functions which need less than 1 page (4096 B = 1000h) have an instruction at the beginning similar to this:
While my debugged function had this instead:
call __chkstk (018104AA00h)
The articles say that when reserving more than one page of stack memory, this function must be called to loop over addresses with 4 KB step and "touch" each page. This is really what it does:
--- f:\dd\vctools\crt\crtw32\startup\amd64\chkstk.asm ---
mov qword ptr [rsp],r10
mov qword ptr [rsp+8],r11
mov r11,qword ptr gs:[10h]
jae cs10+10h (018104AA40h)
mov byte ptr [r11],0
jne cs10 (018104AA30h)
mov r10,qword ptr [rsp]
mov r11,qword ptr [rsp+8]
Key sentence of the second linked article seems to be: "The parameter in rax is size of data you want to add." In my case, eax is set to 26B29h = 158505. Wait, what?! This is more than 150 KB! Is it really how much of the stack the function needs?!
It was finally the right conclusion. The function was more than 3000-lines long, with lots of nested conditions and all kinds of stuff, but mostly an all-encompassing switch with dozens of different cases. I refactored it, extracting code from under each case to a separate function. This fixed the "Stack Overflow" crash.
Apparently if you have a long function and define a lot of local variables, even if they are not particularly big and they are placed inside separate scopes like if-s or switch case-s, the function may need as much as 150 KB of stack frame, at least in Debug configuration. This can cause crash with "Stack Overflow" message even without infinite recursion or bugs like that. So please keep this in mind as additional argument for refactoring your code as soon as you see the need for it.
Global Game Jam 2016 - Postmortem of our project
Last weekend, this year's edition of Global Game Jam took place all around the world. Just like in previous years, I participated in 3City Game Jam - a site in Gdańsk, Poland. It is a big one, with over 150 participants, organized by Playsoft company in their office. Theme this year was "Ritual". Regarding technology, Unity was most popular in our site, with just few games using something else: Unreal Engine, HTML5, GameMaker and C++ with SFML.
We have also used Unity. Our team consisted of 3 programmers. Here you can see our game: Bloody Eclipse, but it is far from being finished or playable. Honestly speaking, in my opinion the project on this jam went exceptionally poor. We didn't even make it to the top 10 best voted games to be presented on a big screen. That's why I'd like to share some conclusions, for you as well as for my future self.
First, it were not environmental issues that caused any problems. We all had our hardware and software set up before the jam, with Unity, Visual Studio, Git client and other tools already in place. Internet worked perfectly with transfer up to 80 Mbps in both directions. Second, it was not a lack of knowledge or skills. Our work in Unity went quite smoothly. We could deal with C#, 3D math and Git pretty well. Third, it was not because of the lack of artists in our team. Sure, graphics is very important for overall experience, but the guys who made The Bad Ritual also didn't have artists in their team and they somehow found a consistent visual style for their game, made it fun and pretty. There are many possibilities to make minimalistic and yet visually pleasant game, just like there are many free assets ready to use in Unity Asset Store.
The biggest thing that was missing in our team was management/leadership. I deliberately don't call it planning or design, because in a hectic environment like a game jam it's not enough to design the game at the beginning and then just execute. Things are changing fast, new ideas come to mind, time is running fast and new obstacles appear (like bugs or difficulties in development), so someone should have an authority to decide what to do next, keep the list of tasks "TODO" and update it constantly with priorities assigned so the most important things are done first. Noone took this role in our team. As the result, we've spent almost whole Saturday developing and polishing algorithm for enemy movement and around half an hour brainstorming and then voting for the game title, while our game used untextured, placeholder cubes and spheres as models until the very end :)
Conclusion: It's not enough to know how to code. It's also important to decide WHAT to code so that best possible result can be achieved with limited time and resources.
But the Global Game Jam as a whole is not a contest (despite our site actually was one, with PlayStation 4 for each team member as first prize) but just a fun, creative event. Despite all the problem we had I think it was fun. I had yet another opportunity to use Unity, which is a great technology. I realized I can handle Git pretty well, despite I don't feel like an expert knowing about "rebase" and such advanced stuff. I realized I still remember how to use the so much unintuitive inteface of Blender, which I learned many years ago to use in my master thesis. I could play many interesting games created on this jam, like my favorite: Witch Rite (it took 3rd place) or the one that won the contest: Acolytes: Ritual of Ascension. And finally, I've met many interesting people who do all sorts of crazy stuff, from running a company that produces medical software and hardware, to visiting escepe rooms and practicing celtic dances :)