In some cases, it may be advantageous to have a task run and, eventually, after it has accomplished everything it needs to, remove it from the system. For example, in some systems with fairly involved startup routines, it might be advantageous to run some of the late initialization code inside a task. In this case, the initialization code would run, but there is no need for an infinite loop. If the task is kept around, it will still have its stack and TCB wasting FreeRTOS heap space. Deleting the task will free the task's stack and TCB, making the RAM available for reuse.
All of the critical initialization code should be run long before the scheduler starts.