About test labels for object detection

[SeongjuLee]

Hello,

Thank you for providing the awesome dataset.

I am trying to build a benchmark for BEV detection; however, I noticed that the test labels are not available. Is this because of the leaderboard restrictions?

If so, how can I set up the test sets for the purpose of an academic paper? Do you consider using the validation set in this case?

[keenan-burnett]

Is this because of the leaderboard restrictions?

Yes

If so, how can I set up the test sets for the purpose of an academic paper?

You can submit to the automated benchmark on our website to get results on the test set.

[SeongjuLee]

Thank you for your reply!

I have another question. Referring to the get_closest_frame function in pyboreas/utils/utils.py, the synchronization tolerance is set to 3.0 seconds. I think this tolerance is a bit large for conducting sensor fusion.
Can you explain the reason why the tolerance is set to 3.0 seconds?

def get_closest_frame(query_time, frame_times, frames):
    """Retrives the closest frame to query_time
    Args:
        query_time (float)
        frame_times (list): list of timestamps which corresponds to the frames list
        frames: (list): list of frames
    Returns:
        closest_frame (SensorType)
    """
    closest = get_closest_index(query_time, frame_times)
    assert abs(query_time - frame_times[closest]) < 3.0, "query: {}".format(query_time)
    return frames[closest]

[keenan-burnett]

Yeah, that's a bit big. You could fork the repo and change it if you like?

[leehungu]

Hello, could you please provide the timestamps for the test dataset? This will allow us to generate corresponding predictions.
@keenan-burnett