It depends on your job profile. If you take the field of technical writing, your knowledge of grammar has to be excellent. You cannot afford to make any kind of grammatical errors. Tecnical Writing is a field wherein you convert complex information and put it in a simple language which a layman can easily understand and comprehend.

At any point of time, if your job profile requires oral as well as written communication, it is imperative for you to be well versed with the nuances of grammar. If you are in a field which requires command over spoken english, then knowledge of grammar is not so critical. We gained fluency over our mother tongue purely by listening to our parents speak and gradually picked up the language. The same principle applies to the English Language also. Another danger of being too conscious about grammar while speaking is the fact that your mind will start analysing each and every word that you speak from a grammatical perspective. This will go on to hamper the communication process.

Tenses form the crux of English Grammar. Make sure you get this right. Also try to interact with people who are proficient in Spoken English and ask them to give you a feedback which will help you correct your structures.

Hi!

In my opinion you can be proficient in spoken English without being "perfect" at the other skills. Especially when it comes to spoken language you have body language which helps you a lot when conveying your ideas. Moreover, if the main goal is communication your audience will not be interested in grammar mistakes but the content of your speech.

So my piece of advice for you is that you do not need to worry too much about tiny details if your objective is communication. Just go with the flow and give it your best shot to make yourself understood.