I try to create forum with permissions:

1. One category with one forum
2. Logged user can create topics 
3. Logged user can see only own topics
4. Guest must see category & forum header (with counters/last post)

1,2,3 - it works. 4 - category don't shows for guests.

When option "Users can see topics posted by other users?" unchecked, i can't change forum permission for guests, checkbox inactive.

Why is such mode impossible? Or did I miss something?

36 minutes ago, desti said:

4. Guest must see category & forum header (with counters/last post)

The first part is possible—I am using that myself for this type of forum. 
However there is no post count and no last poster information, since those are “private information” in this type of forum.  

12 minutes ago, opentype said:

since those are “private information” in this type of forum.

I'm ready not to show the counters, but why I can't show the category and name of the forum??

30 minutes ago, desti said:

I'm ready not to show the counters, but why I can't show the category and name of the forum??

I don’t know. You haven’t shown your category permission. Did you give guest access there? That would be required. If the container doesn’t allow guests, the container content can’t override that. 

Access for all group..

I thought the system should work like this:
the guest cannot have messages, so under the setting "Users can see topics posted by other users? = NO" he simply will not see any topics in the forum. 

I don't understand why this completely disables category/forum display.

It shouldn’t. I use it the way you asked and I have tested it on another installation as well. Maybe open a support ticket so someone can investigate in your ACP directly. 

OpenType, thank you, i'll reinstall the test forum and check.