Hey!
I just wanted to make a quick post saying that this TreadGaming project of mine has been very fun to work with and I had a new sparkle to work towards doing TreadGaming v2.
I'm currently in "research mode", finding hardware and thinking about what to do to make this new version even better and easier than previous version. If you have ideas/suggestions or any feedback, please do let me know over at treadgaming@gmail.com
Use your exercise tool such as a treadmill and a computer to enjoy exercising!
Wednesday, March 27, 2019
Thursday, September 29, 2016
A Map to Perfection: Using D3.js to Make Beautiful Web Maps
Writer's Note: In Game development, effective mapping has proven to be more useful, as it has been a key component of games such as Pokemon Go and the like. This guide would help readers to create maps of their own. Take note, however, that there are various ways to do this though and this is one of those ways.
Data Driven Documents, or D3.js, is “a JavaScript library for manipulating documents based on data”. Or to put it more simply, D3.js is a data visualization library. It was developed by Mike Bostock with the idea of bridging the gap between static display of data, and interactive and animated data visualizations.
D3 is a powerful library with a ton of uses. In this tutorial, I’ll discuss one particularly compelling application of D3: map making. We’ll go through the common challenges of building a useful and informative web map, and show how in each case, D3.js gives capable JavaScript developers everything they need to make maps look and feel beautiful.
What is D3.js used for?
D3.js can bind any arbitrary data to a Document Object Model (DOM), and then, through the use of JavaScript, CSS, HTML and SVG, apply transformations to the document that are driven by that data. The result can be simple HTML output, or interactive SVG charts with dynamic behavior like animations, transitions, and interaction. All the data transformations and renderings are done client-side, in the browser.
At its simplest, D3.js can be used to manipulate a DOM. Here is a simple example where D3.js is used to add a paragraph element to an empty document body, with “Hello World” text:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>D3 Hello World</title>
<script src="http://d3js.org/d3.v3.min.js"></script>
</head>
<body>
<script type="text/javascript">
d3.select("body").append("p").text("Hello World");
</script>
</body>
</html>
The strength of D3.js, however, is in its data visualization ability. For example, it can be used to create charts. It can be used to create animated charts. It can be even used to integrate and animate different connected charts.
D3 for Web Maps and Geographic Data Visualization
But D3.js can be used for much more than just DOM manipulation, or to draw charts. D3.js is extremely powerful when it comes to handling geographical information. Manipulating and presenting geographic data can be very tricky, but building a map with a D3.js is quite simple.
Here is a D3.js example that will draw a world map based on the data stored in a JSON-compatible data format. You just need to define the size of the map and the geographic projection to use (more about that later), define an SVG element, append it to the DOM, and load the map data using JSON. Map styling is done via CSS.
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>D3 World Map</title>
<style>
path {
stroke: white;
stroke-width: 0.5px;
fill: black;
}
</style>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="http://d3js.org/topojson.v0.min.js"></script>
</head>
<body>
<script type="text/javascript">
var width = 900;
var height = 600;
var projection = d3.geo.mercator();
var svg = d3.select("body").append("svg")
.attr("width", width)
.attr("height", height);
var path = d3.geo.path()
.projection(projection);
var g = svg.append("g");
d3.json("world-110m2.json", function(error, topology) {
g.selectAll("path")
.data(topojson.object(topology, topology.objects.countries)
.geometries)
.enter()
.append("path")
.attr("d", path)
});
</script>
</body>
</html>Geographic Data for D3
For this D3.js tutorial, keep in mind that map building works best with data formatted in JSON formats, particularly the GeoJSON and TopoJSON specifications.
GeoJSON is “a format for encoding a variety of geographic data structures”. It is designed to represent discrete geometry objects grouped into feature collections of name/value pairs.
TopoJSON is an extension of GeoJSON, which can encode topology where geometries are “stitched together from shared line segments called arcs”. TopoJSON eliminates redundancy by storing relational information between geographic features, not merely spatial information. As a result, geometry is much more compact and combined where geometries share features. This results with 80% smaller typical TopoJSON file than its GeoJSON equivalent.
So, for example, given a map with several countries bordering each other, the shared parts of the borders will be stored twice in GeoJSON, once for each country on either side of the border. In TopoJSON, it will be just one line.
Map Libraries: Google Maps and Leaflet.js
Today, the most popular mapping libraries are Google Maps and Leaflet. They are designed to get “slippy maps” on the web fast and easy. “Slippy maps” is a term referring to modern JavaScript-powered web maps that allow zooming and panning around the map.
Leaflet is a great alternative to Google Maps. It is an open source JavaScript library designed to make mobile-friendly interactive maps, with simplicity, performance and usability in mind. Leaflet is at its best when leveraging the big selection of raster-based maps that are available around the internet, and brings the simplicity of working with tiled maps and their presentation capabilities.
Leaflet can be used with great success when combined with D3.js’s data manipulation features, and for utilizing D3.js for vector based graphics. Combining them together brings out the best in both libraries.
Google Maps are more difficult to combine with D3.js, since Google Maps are not open source. It is possible to use Google Maps and D3 together, but this is mostly limited to overlaying data with D3.js over Google Maps background maps. Deeper integration is not really possible, without hacking.
Projections - Beyond Spherical Mercator
The question of how to project maps of the 3-dimensional spherical Earth onto 2-dimensional surfaces is an old and complex problem. Choosing the best projection for a map is an important decision to make for every web map.
In our simple world map D3.js tutorial above, we used the Spherical Mercator projection coordinate system by calling
d3.geo.mercator(). This projection is also known as Web Mercator. This projection was popularized by Google when they introduced Google Maps. Later, other web services adopted the projection too, namelyOpenStreetMap, Bing Maps, Here Maps and MapQuest. This has made Spherical Mercator a very popular projection for online slippy maps.
All mapping libraries support the Spherical Mercator projection out of the box. If you want to use other projections, you will need to use, for example, the Proj4js library, which can do any transformation from one coordinate system to another. In the case of Leaflet, there is a Proj4Leaflet plugin. In the case of Google Maps, there is, well, nothing.
D3.js brings cartographic projections to a whole new level with built-in support for many different geographic projections. D3.js models geographic projections as full geometric transformations, which means that when straight lines are projected to curves, D3.js applies configurable adaptive resampling to subdivide lines and eliminate projection artifacts. The Extended Geographic Projections D3 plugin brings the number of supported projections to over 40. It is even possible to create a whole new custom projection using
d3.geo.projection and d3.geo.projectionMutator.Raster Maps
As mentioned before, one of the main strengths of D3.js is in working with vector data. To use raster data there is an option to combine D3.js with Leaflet. But there is also an option to do everything with just D3.js using d3.geo.tile to create slippy maps. Even with just D3.js alone, people are doing amazing things with raster maps.
Vector Manipulation on the Fly
One of the biggest challenges in classic cartography is map generalization. You want to have as much detailed geometry as you can, but that data needs to adapt to the scale of the displayed map. Having too high a data resolution increases download time and slows down rendering, while too low a resolution ruins details and topological relations. Slippy maps using vector data can run into a big problem with map generalization.
One option is to do map generalization beforehand: to have different datasets in different resolutions, and then display the appropriate dataset for the current selected scale. But this multiplies datasets, complicates data maintenance, and is prone to errors. Yet most mapping libraries are limited to this option.
The better solution is to do map generalization on the fly. And here comes D3.js again, with its powerful data manipulation features. D3.js enables line simplification to be done in browser.
I want more!
D3.js is not easy to master and it has a steep learning curve. It is necessary to be familiar with a lot of technologies, namely JavaScript objects, the jQuery chaining syntax, SVG and CSS, and of course D3’s API. On top of that, one needs to have a bit of design skill to create nice graphics in the end. Luckily, D3.js has a big community, and there are a lot of resources for people to dig into. A great starting point for learning D3 isthese tutorials.
If you like learning by examining examples, Mike Bostock has shared more than 600 D3.js examples on his webpage. All D3.js examples have git repository for version control, and are forkable, cloneable and commentable.
If you are using CartoDB, you’ll be glad to hear that CartoDB makes D3 maps a breeze.
And for a little bonus at the end, here’s one of my favorite examples showing off the amazing things D3 is capable of:
- earth, a global animated 3D wind map of the entire world made with D3.js. Earth is a visualization of global weather conditions, based on weather forecasts made by supercomputers at the National Centers for Environmental Prediction, NOAA / National Weather Service and converted to JSON. You can customize displayed data such as heights for the wind velocity readings, change overlaid data, and even change Earth projection.
Source: Toptal
Wednesday, September 21, 2016
The 10 Most Common Mistakes That Unity Developers Make
Unity is a great and straightforward tool to use for multi-platform development. Its principles are easy to understand, and you can intuitively begin to create your products. However, if some things are not taken in mind, they’ll slow your progress when you proceed with your work to the next level, as you are moving from initial prototype phase or approaching a final release. This article will provide advice on how to overcome most common problems and how to avoid fundamental mistakes in your new or existing projects. Please note, the perspective of this article is focused more on 3D application development, but everything mentioned is applicable for 2D development as well.
Unity is a great and straightforward tool to use for multi-platform development.Common Unity Mistake #1: Underestimating Project Planning Phase
For every project, it’s crucial to determine several things before the application design and programming part of the project even begins. These days, when product marketing is an important part of the whole process, it’s also important to have a clear idea of what the business model of the implemented application will be. You have to be sure what platforms you will be releasing product for, and what platforms are in your plan. It’s also necessary to set the minimal supported devices specifications (will you support older low-end devices or just more recent models?) to have the idea of what performance and visuals you can afford. Every topic in this article is influenced by this fact.
From a more technical point of view, it should be necessary to set in advance the whole workflow of creating assets and models while providing them to the programmer, with particular attention to iteration process when the models will need some more changes and refinements. You should have a clear idea about desired frame rate and vertex budget, so the 3D artist can know in what maximal resolution the models have to be, and how many LOD variations he has to do. It should also be specified how to unify all the measurements to have a consistent scale, and import process throughout the whole application.
The way the levels will be designed is crucial for future work because the division of the level influences the performance a lot. Performance concerns have to always be in your mind when designing new levels. Don’t go with unrealistic visions. It’s always important to ask yourself the question “can it be reasonably achieved?” If not, you shouldn’t waste your precious resources on something hardly achievable (in case it’s not part of your business strategy to have it as your main competitive advantage, of course).
Common Unity Mistake #2: Working with Unoptimized Models
It’s critical to have all your models well prepared to be able to use them in your scenes without further modifications. There are several things that the good model should fulfill.
It’s important to set the scale correctly. Sometimes it’s not possible to have this correctly set from your 3D modeling software because of different units these applications are using. To make everything right, set the scale factor in models import settings (leave 0.01 for 3dsMax and Modo, set 1.0 for Maya), and note that sometimes you will need to re-import objects after changing the scale setting. These settings should assure that you can use just basic scale 1,1,1 in your scenes to get consistent behavior and no physics problems. Dynamic batching will also more likely work correctly. This rule should also be applied on every subobject in the model, not just the main one. When you need to tweak object dimensions, do it with regards to other objects in 3D modeling application rather than in Unity. However, you can experiment with scale in Unity to find out appropriate values, but for the final application and consistent workflow, it’s good to have everything well prepared before importing to Unity.
Regarding object’s functionality and their dynamic parts - have your models well divided. The fewer subobjects, the better. Separate parts of the object just in case when you need them, for example, to move or rotate dynamically, for animation purposes, or other interactions. Every object and its subobjects should have its pivot properly aligned and rotated with regards to its main function. The main object should have the Z axis pointing forward and pivot should be at the bottom of the object for better placement to the scene. Use as few materials on objects as possible (more on this below).
All assets should have proper names which will easily describe its type and functionality. Keep this consistency throughout all of your projects.
Common Unity Mistake #3: Building Interdependent Code Architecture
Prototyping and implementation of functionality in Unity is quite easy. You can easily drag and drop any references to other objects, address every single object in the scene, and access every component it has. However, this can also be potentially dangerous. On top of noticeable performance issues (finding an object in the hierarchy and access to components has its overhead), there is also great danger in making parts of your code entirely dependent on each other. Or being dependent on other systems and scripts unique to your application, or even on the current scene, or current scenario. Try to take a more modular approach and create reusable parts which can be used in other parts of your application, or even shared across your whole application portfolio. Build your framework and libraries on top of Unity API the same way you are building your knowledge base.
There’s a lot of different approaches to ensure this. A good starting point is the Unity component system itself. Complications may appear when particular components need to communicate with other systems of the application. For this, you can use interfaces to make parts of your system more abstract and reusable. Alternatively, you can use an event-driven approach for reacting to particular events from outside scope, either by creating a messaging system or by registering directly to parts of the other system as listeners. The right approach will be trying to separate gameObject properties from program logic (at least something like model-controller principle), because it’s tough to identify which objects are modifying its transform properties, like position and rotation. It should be exclusively its controller’s responsibility.
Try to make everything well documented. Treat it always like you should return to your code after a long time and you need to understand quickly what exactly this part of the code is doing. Because in reality, you will quite often get to some parts of your application after some time and it’s an unnecessary obstacle for quickly jumping into the problem. But do not overdo this. Sometimes, an appropriate class, method or property name is quite sufficient.
Common Unity Mistake #4: Wasting Your Performance
The latest product line of mobile phones, consoles, or desktop computers will never be so advanced that there will be no need to care about performance. Performance optimizations are always needed, and they provide the foundation for making the difference in how your game or application looks like in comparison to others on the market. Because when you save some performance in one part, you can use that to polish other parts of your application.
There are a lot of areas for optimizations. The whole article would be needed just to scratch the surface about this topic. At least, I will try to divide this domain into some core areas.
Update Loops
Don’t use performance intensive things in update loops, use caching instead. A typical example is an access to components or other objects in a scene or intensive calculations in your scripts. If possible, cache everything in
Awake() methods, or change your architecture to a more event-driven approach to trigger things just when they’re needed.Instantiations
For objects that are instantiated quite often (for example, bullets in an FPS game), make a pre-initialized pool of them and just pick one already initialized when you need it and activate it. Then, instead of destroying it when it is not required anymore, deactivate it and return it to the pool.
Rendering
Use occlusion culling or LOD techniques to limit rendered parts of the scene. Try to use optimized models to be able to keep vertex count in the scene under control. Be aware, vertex count isn’t just the number of vertices on the model itself, but it’s influenced by other things like normals (hard edges), UV coordinates (UV seams) and vertex colors. Also, a number of dynamic lights in the scene will dramatically influence overall performance, so try to bake everything in advance whenever possible.
Draw Calls
Try to reduce draw calls count. In Unity, you can achieve draw calls reduction by using static batching for still objects and dynamic batching for the moving ones. However, you have to prepare your scenes and models first (batched objects have to share same materials), and batching of dynamic objects works only for low-res models. Alternatively, you could combine meshes by the script into one (
Mesh.CombineMeshes) instead of using batching, but you have to be careful not to create too large objects which can’t take advantage of view frustum culling on some platforms. In general, the key is to use as little materials as possible and share them across the scene. You will sometimes need to create atlases from textures to be able to share one material between distinct objects. A good tip is also to use higher resolution of scene lightmaps textures (not generated resolution, but texture output resolution) to lower their number when you are baking light in larger environments.Overdraw Problems
Don’t use transparent textures when not necessary, as it will cause fill-rate problems. It is okay to use it for complicated and more distant geometry, like trees or bushes. When you need to use it, prefer alpha blended shaders instead of shaders with alpha testing or instead of cutout shaders for mobile platforms. For identifying these problems in general, try to lower the resolution of your application. If it helps, it might be possible that you have these fill-rate problems, or you need to optimize your shaders more. Otherwise, it can be a more memory problem.
Shaders
Optimize your shaders for better performance. Reduce the number of passes, use variables with lower precision, replace complicated math calculations with pre-generated lookup textures.
Always use a profiler to determine the bottlenecks. It’s a great tool. For rendering, you can also use awesome Frame Debugger, which will help you learn a lot about how things work in general when decomposing rendering processes with it.
Common Unity Mistake #5: Ignoring Garbage Collection problems
It is necessary to realize that despite the fact that Garbage Collector (GC) itself helps us to be really efficient and focused on important things in programming, there are a few things we should be explicitly aware of. The usage of GC isn’t free. Generally, we should avoid unnecessary memory allocations to prevent GC from firing itself too often and thus spoiling performance by framerate spikes. Ideally, there shouldn’t be any new memory allocations happening regularly each frame at all. However, how can we achieve this goal? It’s really determined by application architecture, but there are some rules you could follow which help:
- Avoid unnecessary allocations in update loops.
- Use structs for simple property containers, as they’re not allocated on the heap.
- Try to preallocate arrays or lists or other collections of objects, instead of creating them inside update loops.
- Avoid using mono problematic things (like LINQ expressions or foreach loops, for example) because Unity is using an older, not ideally optimized version of Mono (at the time of writing it is modified version 2.6, with upgrade on the roadmap).
- Cache strings in
Awake()methods or in events. - If update of string property in update loop is necessary, use StringBuilder object instead of string.
- Use profiler to identify potential problems.
Common Unity Mistake #6: Optimizing Memory and Space Usage Last
It is necessary to keep the attention on the lowest memory and space usage of the application from the beginning of the project, as it is more complicated to do it when you leave optimization for pre-release phase. On mobile devices, this is even more important, because we are quite short on resources there. Also, by exceeding 100MB size of the installation, we can lose a significant amount of our customers. This is because of the 100MB limit for cellular network downloads, and also because of psychological reasons. It is always better when your application doesn’t waste customer’s precious phone resources, and they will be more likely to download or buy your app when its size is smaller.
For finding resource drainers, you can use editor log where you can see (after every new build) the size of resources divided into separate categories, like audio, textures, and DLLs. For better orientation, there are editor extensions on the Unity Asset Store, which will provide you a detailed summary with referenced resources and files in your filesystem. Actual memory consumption can also be seen in the profiler, but it is recommended to test it when connected to build on your target platform because there are a lot of inconsistencies when testing in an editor or on anything other than your target platform.
The biggest memory consumers are often textures. Preferably, use compressed textures as they take much less space and memory. Make all textures squared, ideally, make the length of both sides power of two (POT), but keep in mind Unity can also scale NPOT textures to POT automatically. Textures can be compressed when being in the POT form. Atlas textures together to fill the whole texture. Sometimes you can even use texture alpha channel for some extra information for your shaders to save additional space and performance. And of course, try to reuse textures for your scenes as much as possible, and use repeating textures when it is possible to retain good visual appearance. For low-end devices, you can lower the resolution of textures in Quality Settings. Use compressed audio format for longer audio clips, like the background music.
When you are dealing with different platforms, resolutions or localizations, you can use asset bundles for using different sets of textures for different devices or users. These asset bundles can be loaded dynamically from the internet after the application was installed. This way, you can exceed the 100MB limit by downloading resources during the game.
Common Unity Mistake #7: Common Physics Mistakes
Sometimes, when moving objects in the scene, we don’t realize that the object has a collider on it and that changing its position will force the engine to recalculate the whole physical world all over again. In that case, you should add
Rigidbody component to it (you can set it to non-kinematic if you don’t want external forces to be involved).
To modify the position of the object with
Rigidbody on it, always set Rigidbody.position when a new position doesn’t follow the previous one, or Rigidbody.MovePosition when it is a continuous movement, which also takes interpolation into account. When modifying it, apply operations always in FixedUpdate, not in Updatefunctions. It will assure consistent physics behaviors.
If possible, use primitive colliders on gameObjects, like sphere, box, or cylinder, and not mesh colliders. You can compose your final collider from more than one of these colliders. Physics can be a performance bottleneck of your application because of its CPU overhead and collisions between primitive colliders are much faster to calculate. You can also adjust Fixed Timestep setting in Time manager to reduce the frequency of physics fixed updates when the accuracy of physics interaction isn’t so necessary.
Common Unity Mistake #8: Manually Testing All Functionality
Sometimes there might be a tendency to test functionality manually by experimenting in the play mode because it is quite fun and you have everything under your direct control. But this cool factor can decrease quite quickly. The more complex the application becomes, the more tedious tasks the programmer has to repeat and think about to assure that the application behaves as it was originally intended. It can easily become the worst part of the whole development process, because of its repetitive and passive character. Also, because the manual repetition of testing scenarios isn’t that fun, so there’s a higher chance that some bugs will make it through the whole testing process.
Unity has great testing tools to automate this. With appropriate architectural and code design, you can use unit tests for testing isolated functionality, or even integration tests for testing more complex scenarios. You can dramatically reduce try-and-check approach where you’re logging actual data and comparing it with its desired state.
Manual testing is without a doubt a critical part of the development. But its amount can be reduced, and the whole process can be more robust and faster. When there’s no possible way to automate it, prepare your test scenes to be able to get into the problem you are trying to solve as quickly as possible. Ideally, a few frames after hitting the play button. Implement shortcuts or cheats to set the desired state for testing. Also, make the testing situation isolated to be sure what’s causing the problem. Every unnecessary second in the play mode when testing is accumulated, and the bigger the initial bias of testing the problem, the more likely you won’t test the problem at all, and you will hope that all works just fine. But it probably won’t.
Common Unity Mistake #9: Thinking Unity Asset Store Plugins Will Solve All Your Problems
Trust me; they won’t. When working with some clients, I sometimes faced the tendency or relicts from the past of using asset store plugins for every single little thing. I don’t mean there aren’t useful Unity extensions on the Unity Asset Store. There are many of them, and sometimes it’s even hard to decide which one to choose. But for every project, it is important to retain consistency, which can be destroyed by unwisely using different pieces that do not fit well together.
On the other hand, for functionality which would take a long time for you to implement, it is always useful to use well-tested products from Unity Asset Store, which can save you a huge amount of your development time. However, pick carefully, use the proven ones which won’t bring a lot of uncontrollable and weird bugs to your final product. Five-star reviews are a good measure for a start.
If your desired functionality isn’t hard to implement, just add it to your constantly growing personal (or company’s) libraries, which can be used in all of your projects later again. That way you are improving your knowledge and your toolset at the same time.
Common Unity Mistake #10: Having No Need to Extend Unity Basic Functionality
Sometimes it may seem that Unity Editor environment is quite sufficient for basic game testing and level design, and extending it is a waste of time. But trust me, it is not. Unity’s great extension potential comes from being able to adapt it to specific problems which need to be solved in various projects. This can either improve the user experience when working in Unity or dramatically speed up the entire development and level design workflow. It would be unfortunate not to use built-in features, like built-in or custom Property Drawers, Decorator Drawers, custom component inspector settings, or even to not build whole plugins with its own Editor Windows.
Conclusion
I hope these topics will be useful for you as you move your Unity projects further. There are a lot of things which are project specific, so they can’t be applied, but it’s always useful to have some basic rules in mind when trying to solve more difficult and specific problems. You might have different opinions or procedures on how to solve these problems in your projects. The most important is to keep your idioms consistent throughout your project so that anyone in your team can clearly understand how the particular domain should have been solved correctly.
Subscribe to:
Posts (Atom)