Job ID: 25558 
Pay Rate: $54 Per HR  
Location: Remote in WA 
Employment Type: Contract 
Schedule: Monday – Friday, 8 AM – 5 PM  
  
Our client is passionate about connecting people with the homes of their dreams. Named as one of Seattle’s “best place to work”, this company has disrupted the real estate industry with its innovative, forward-thinking approach to buying or renting a home. Whether you’ve bought a home or you’ve worked in the Real Estate industry before, you know how challenging this environment can be and how quickly marketing strategies need to pivot in order to hold the consumers attention long enough.   

As the Agentic Judge – Real Estate, you will evaluate annotated data and AI-generated outputs to ensure quality, accuracy, and alignment with project objectives. You will collaborate with engineering and data science teams to establish ground truth data, refine evaluation standards, improve training content, and provide actionable feedback that enhances the performance of agentic AI systems within the real estate domain. 

Essential Duties and Responsibilities  

• Review and evaluate annotated real estate data to ensure quality and accuracy standards are met. 

• Provide feedback to engineering teams to improve AI model performance and output quality. 

• Collaborate with engineers and data scientists to optimize models using evaluation results and insights. 

• Develop and maintain evaluation guidelines, quality standards, and best practices. 

• Create and validate ground truth datasets for training and benchmarking purposes. 

• Provide subject matter expertise, examples, and input for training content development. 

• Identify gaps in training materials and recommend improvements to enhance model effectiveness. 

• Document findings, trends, and recommendations to support continuous improvement initiatives. 

• Other duties as assigned  

Qualifications  

• Active real estate license with 3+ years of experience as a real estate agent or broker. 

• Strong understanding of the real estate transaction lifecycle and industry practices. 

• Demonstrated interest in artificial intelligence and experience evaluating AI-generated content. 

• Bachelor’s degree in English, Linguistics, Communications, or a related field. 

• Experience supporting Natural Language Processing (NLP) initiatives, including transcription, localization, translation, or content evaluation. 

• Excellent analytical, critical thinking, and attention-to-detail skills. 

• Ability to work effectively in a collaborative, cross-functional environment. 

• Proficiency with annotation, evaluation, or data-labeling tools is preferred. 

• Experience with content development, copywriting, UX writing, or voice and brand guidelines is a plus. 

• Familiarity with machine-generated speech evaluation, search result evaluation, or search query categorization is preferred. 

• Ability to follow complex processes, exercise sound judgment, and work independently while contributing to team goals. 

Benefits offered by Parker Staffing to Contract Employees: 

• Paid Sick/Safe Time (based on where allowable by law)    

• Medical benefits with multiple plan tiers and Telemedicine access    

• 401K with matching contributions     

• Holiday pay     

• Employee Assistance Program (EAP)  

Parker Staffing offers all levels of Administrative, Customer Service, Call Center, Sales, and Human Resources job opportunities in the Seattle and Bellevue metro area and other large cities throughout the Nation. Parker Staffing has served as the staffing agency of choice for thousands for over four decades. Visit our employment opportunities page at http://www.parkerstaffing.com to review our full offering of temp, temp-to-hire, and direct hire job openings!  

Smoking/vaping and the use of tobacco products are prohibited on all Company premises, including indoor and outdoor areas, parking lots, and Company-owned vehicles. 

As part of our employment process, candidates who received a conditional offer may be required to undergo pre-employment drug testing. 

Parker Staffing is an Equal Opportunity Employer and does not discriminate based on race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other protected status.