can you orderby a grid field on exp:channel:entries?

I have a users table with multiple grid fields based off week you are on. I want to list all users in order based off a grid field called “total_wins”. Is this possible? I need to access fields from each entry outside of the grid so I don’t think i want to put the orderby on the grid pair.

I’ve tried something like this for the orderby

{exp:channel:entries 
    channel="users" 
 search:league_id="{segment_2}"
 orderby="week_{segment_4}_picks:total_wins" 
 sort="desc"
 disable="categories|category_fields|pagination|relationships|relationship_custom_fields|relationship_categories"
{/exp:channel:entries}

i’ve also tried orderby=”total_wins” and orderby=”{week_{segment_4}picks}{week{segment_4}picks:total_wins}{/week{segment_4}_picks}”