Ideally, you will rarely get to a point where you need to build your own operator from scratch by implementing ObservableOperator or FlowableOperator. ObservableTransformer and FlowableTransformer will hopefully satisfy most cases where you can use existing operators to compose new ones, and this is usually the safest route. But on occasion, you may find yourself having to do something that the existing operators cannot do or not do easily. After you exhaust all other options, you may have to create an operator that manipulates each onNext(), onComplete(), and onError() event between the upstream and the downstream.
Before you go out and create your own operator, try to use existing operators first with compose() and a Transformer. After that fails, it is recommended that you post a question on StackOverflow and ask the RxJava community whether such an operator exists...