Graphical Abstract Figure
Graphical Abstract Figure
Close modal

Abstract

The crowdsourcing literature has shown that domain experts are not always the best solvers for complex system design problems. Under certain conditions, novices and specialists in adjacent domains can provide novel solutions at lower costs. Additionally, the best types of solvers for different problems are dependent on the architecture of complex systems. The joint consideration of solver assignment and system decomposition, referred to as solver-aware system architecting (SASA), expands traditional system architecting practices by considering solver characteristics and contractual incentive mechanisms in the design process and aims to improve complex system design and innovation by leveraging the strengths of domain experts, crowds, and specialists for different parts of the problem. The joint consideration of problem decomposition and solver assignment decisions in SASA renders the design space exponentially more complex. Therefore, new computationally efficient and mathematically rigorous methods are needed to explore this high-dimensional space and extract reliable heuristics. To address this need, this paper presents a computational approach using a Markov decision process (MDP) formulation, Q-learning, and Gaussian mixture models. Together, these techniques explore the large space of possible solver–module assignments by modeling the sequential nature of solver assignment decisions, capturing these temporal dependencies, thereby enabling optimization for long-term expected rewards, and analyzing reward distributions. The approach identifies heuristics for solver assignment based on the designer’s preference for cost-performance trade-off through the parameterized reward function. The approach is demonstrated using a simple and idealized golf problem, which has characteristics similar to design problems, including how the problem is decomposed into interdependent modules and can be solved by different solvers with different strengths that interact with the module type. The results show that the proposed approach effectively elicits a rich set of heuristics applicable in various contexts for the golf problem and can be extended to more complex systems design problems.

References

1.
Carlile
,
P. R.
,
2004
, “
Transferring, Translating, and Transforming: An Integrative Framework for Managing Knowledge Across Boundaries
,”
Organ. Sci.
,
15
(
5
), pp.
555
568
.
2.
Vincenti
,
W. G.
,
1990
,
What Engineers Know and How They Know It
,
Johns Hopkins University Press
,
Baltimore, MD
.
3.
Cappelli
,
P. H.
,
2015
, “
Skill Gaps, Skill Shortages, and Skill Mismatches: Evidence and Arguments for the United States
,”
ILR Rev.
,
68
(
2
), pp.
251
290
.
4.
Leonardi
,
P. M.
,
2011
, “
Innovation Blindness: Culture, Frames, and Cross-Boundary Problem Construction in the Development of New Technology Concepts
,”
Organ. Sci.
,
22
(
2
), pp.
347
369
.
5.
Chesbrough
,
H. W.
,
2003
, “
A Better Way to Innovate
,”
Harvard Bus. Rev.
,
81
(
7
), pp.
12
13
.
6.
Gambardella
,
A.
,
Raasch
,
C.
, and
von Hippel
,
E.
,
2016
, “
The User Innovation Paradigm: Impacts on Markets and Welfare
,”
Manage. Sci.
,
63
(
5
), pp.
1450
1468
.
7.
Panchal
,
J. H.
,
2015
, “
Using Crowds in Engineering Design—Towards a Holistic Framework
,” DS 80-8 Proceedings of the 20th International Conference on Engineering Design (ICED 15), Vol. 8: Innovation and Creativity, Milan, Italy, July 27–30, pp.
41
50
.
8.
Chaudhari
,
A. M.
,
Sha
,
Z.
, and
Panchal
,
J. H.
,
2018
, “
Analyzing Participant Behaviors in Design Crowdsourcing Contests Using Causal Inference on Field Data
,”
ASME J. Mech. Des.
,
140
(
9
), p.
091401
.
9.
Szajnfarber
,
Z.
, and
Weigel
,
A. L.
,
2013
, “
A Process Model of Technology Innovation in Governmental Agencies: Insights From NASA’s Science Directorate
,”
Acta Astronaut.
,
84
, pp.
56
68
. Please provide missing issue for Refs. 9, 10, 13, and 33.
10.
Vrolijk
,
A.
, and
Szajnfarber
,
Z.
,
2015
, “
When Policy Structures Technology: Balancing Upfront Decomposition and In-Process Coordination in Europe’s Decentralized Space Technology Ecosystem
,”
Acta Astronaut.
,
106
, pp.
33
46
.
11.
Maier
,
M. W.
, and
Rechtin
,
E.
,
2009
,
The Art of Systems Architecting
,
CRC Press
,
Boca Raton, FL
.
12.
Crawley
,
E.
,
Cameron
,
B.
, and
Selva
,
D.
,
2015
,
System Architecture: Strategy and Product Development for Complex Systems
,
Prentice Hall Press
,
Upper Saddle River, NJ
.
13.
Szajnfarber
,
Z.
,
Topcu
,
T. G.
, and
Lifshitz-Assaf
,
H.
,
2022
, “
Towards a Solver-Aware Systems Architecting Framework: Leveraging Experts, Specialists and the Crowd to Design Innovative Complex Systems
,”
Des. Sci.
,
8
, p.
e10
.
14.
Szajnfarber
,
Z.
,
Grogan
,
P. T.
,
Panchal
,
J. H.
, and
Gralla
,
E. L.
,
2020
, “
A Call for Consensus on the Use of Representative Model Worlds in Systems Engineering and Design
,”
Syst. Eng.
,
23
(
4
), pp.
436
442
.
15.
Chaudhari
,
A. M.
,
Gralla
,
E. L.
,
Szajnfarber
,
Z.
,
Grogan
,
P. T.
, and
Panchal
,
J. H.
,
2020
, “
Designing Representative Model Worlds to Study Socio-technical Phenomena: A Case Study of Communication Patterns in Engineering Systems Design
,”
ASME J. Mech. Des.
,
142
(
12
), p.
121403
.
16.
Hazelrigg
,
G.
,
1998
, “
A Framework for Decision-Based Engineering Design
,”
ASME J. Mech. Des.
,
120
(
4
), pp.
653
658
.
17.
Collopy
,
P. D.
, and
Hollingsworth
,
P. M.
,
2011
, “
Value-Driven Design
,”
J. Aircr.
,
48
(
3
), pp.
749
759
.
18.
Topcu
,
T. G.
, and
Mesmer
,
B. L.
,
2018
, “
Incorporating End-User Models and Associated Uncertainties to Investigate Multiple Stakeholder Preferences in System Design
,”
Res. Eng. Des.
,
29
(
3
), pp.
411
431
.
19.
Hennig
,
A.
,
Topcu
,
T. G.
, and
Szajnfarber
,
Z.
,
2022
, “
So You Think Your System Is Complex?: Why and How Existing Complexity Measures Rarely Agree
,”
ASME J. Mech. Des.
,
144
(
4
), p.
041401
.
20.
Sutton
,
R. S.
, and
Barto
,
A. G.
,
2018
,
Reinforcement Learning: An Introduction
,
MIT Press
,
Cambridge, MA
.
21.
Haskins
,
C.
,
Forsberg
,
K.
, and
Krueger
,
M.
,
2011
,
Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities
,
Incose
,
San Diego, CA
.
22.
Fixson
,
S. K.
, and
Park
,
J.-K.
,
2008
, “
The Power of Integrality: Linkages Between Product Architecture, Innovation, and Industry Structure
,”
Res. Policy
,
37
(
8
), pp.
1296
1316
.
23.
O’Neill
,
M. G.
, and
Weigel
,
A. L.
,
2011
, “
Assessing Fractionated Spacecraft Value Propositions for Earth Imaging Space Missions
,”
J. Spacecraft Rockets
,
48
(
6
), pp.
974
986
.
24.
Ulrich
,
K.
,
1995
, “
The Role of Product Architecture in the Manufacturing Firm
,”
Res. Policy
,
24
(
3
), pp.
419
440
.
25.
Brusoni
,
S.
, and
Prencipe
,
A.
,
2001
, “
Unpacking the Black Box of Modularity: Technologies, Products and Organizations
,”
Ind. Corporate Change
,
10
(
1
), pp.
179
205
.
26.
Holmqvist
,
T. K. P.
, and
Persson
,
M. L.
,
2003
, “
Analysis and Improvement of Product Modularization Methods: Their Ability to Deal With Complex Products
,”
Syst. Eng.
,
6
(
3
), pp.
195
209
.
27.
Topcu
,
T. G.
,
Mukherjee
,
S.
,
Hennig
,
A. I.
, and
Szajnfarber
,
Z.
,
2022
, “
The Dark Side of Modularity: How Decomposing Problems Can Increase System Complexity
,”
ASME J. Mech. Des.
,
144
(
3
), p.
031403
.
28.
Boas
,
R.
,
Cameron
,
B. G.
, and
Crawley
,
E. F.
,
2013
, “
Divergence and Lifecycle Offsets in Product Families With Commonality
,”
Syst. Eng.
,
16
(
2
), pp.
175
192
.
29.
Ross
,
A. M.
,
Rhodes
,
D. H.
, and
Hastings
,
D. E.
,
2008
, “
Defining Changeability: Reconciling Flexibility, Adaptability, Scalability, Modifiability, and Robustness for Maintaining System Lifecycle Value
,”
Syst. Eng.
,
11
(
3
), pp.
246
262
.
30.
Mosleh
,
M.
,
Ludlow
,
P.
, and
Heydari
,
B.
,
2016
, “
Distributed Resource Management in Systems of Systems: An Architecture Perspective
,”
Syst. Eng.
,
19
(
4
), pp.
362
374
.
31.
Mosleh
,
M.
,
Dalili
,
K.
, and
Heydari
,
B.
,
2018
, “
Distributed or Monolithic? A Computational Architecture Decision Framework
,”
IEEE Syst. J.
,
12
(
1
), pp.
125
136
.
32.
Fu
,
K. K.
,
Yang
,
M. C.
, and
Wood
,
K. L.
,
2016
, “
Design Principles: Literature Review, Analysis, and Future Directions
,”
ASME J. Mech. Des.
,
138
(
10
), p.
101103
.
33.
Yilmaz
,
S.
,
Daly
,
S. R.
,
Seifert
,
C. M.
, and
Gonzalez
,
R.
,
2015
, “
How Do Designers Generate New Ideas? Design Heuristics Across Two Disciplines
,”
Des. Sci.
,
1
, p.
e4
.
34.
Yilmaz
,
S.
, and
Seifert
,
C. M.
,
2011
, “
Creativity Through Design Heuristics: A Case Study of Expert Product Design
,”
Des. Stud.
,
32
(
4
), pp.
384
415
.
35.
Fillingim
,
K. B.
,
Nwaeri
,
R. O.
,
Borja
,
F.
,
Fu
,
K.
, and
Paredis
,
C. J. J.
,
2020
, “
Design Heuristics: Extraction and Classification Methods With Jet Propulsion Laboratory’s Architecture Team
,”
ASME J. Mech. Des.
,
142
(
8
), p.
081101
.
36.
Deshmukh
,
A. P.
,
Thurston
,
D. L.
, and
Allison
,
J. T.
,
2016
, “
Heuristics for Formulating Design Optimization Models: Their Uses and Pitfalls
,”
CESUN 2016
,
Washington, DC
,
June 27–29
.
37.
Ororbia
,
M. E.
, and
Warn
,
G. P.
,
2023
, “
Design Synthesis of Structural Systems as a Markov Decision Process Solved With Deep Reinforcement Learning
,”
ASME J. Mech. Des.
,
145
(
6
), p.
061701
.
38.
Steward
,
D. V.
,
1981
, “
The Design Structure System: A Method for Managing the Design of Complex Systems
,”
IEEE Trans. Eng. Manage.
,
EM-28
(
3
), pp.
71
74
.
39.
Browning
,
T. R.
,
2016
, “
Design Structure Matrix Extensions and Innovations: A Survey and New Opportunities
,”
IEEE Trans. Eng. Manage.
,
63
(
1
), pp.
27
52
.
40.
Parnas
,
D. L.
,
1972
, “
On the Criteria to Be Used in Decomposing Systems Into Modules
,”
Pioneers Their Contrib. Softw. Eng.
,
15
(
12
), pp.
1053
1058
.
41.
Ulrich
,
K. T.
,
Eppinger
,
S.D.
, and
Yang
,
M.C.
,
2020
,
Product Design and Development
, 7th ed.,
McGraw-Hill Education
,
New York
.
42.
Baldwin
,
C. Y.
, and
Clark
,
K. B.
,
2000
,
Design Rules: The Power of Modularity
, Vol. 1,
MIT Press
,
Cambridge, MA
.
43.
Taylor
,
C. R.
,
1995
, “
Digging for Golden Carrots: An Analysis of Research Tournaments
,”
Am. Econ. Rev.
,
85
(
4
), pp.
872
890
.
44.
Jin
,
C.
,
Allen-Zhu
,
Z.
,
Bubeck
,
S.
, and
Jordan
,
M. I.
,
2018
, “
Is Q-Learning Provably Efficient?
NeurIPS 2018
,
Montreal, Canada
,
Dec. 2–8
.
45.
Dharmarajan
,
A.
,
Topcu
,
T.
,
Panchal
,
J.
, and
Szajnfarber
,
Z.
,
2024
, “
Valuing Outliers: A Modeling Framework to Consider Non-Traditional Solutions From Non-traditional Solvers
,” ASME 2024 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE 2024), Paper No. IDETC2024-143509.
46.
Gadi
,
V.
,
Szajnfarber
,
Z.
, and
Panchal
,
J.
,
2024
, “
Developing Heuristics for Resource Allocation and Utilization in Systems Design: A Hierarchical Reinforcement Learning Approach
,” ASME 2024 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC/CIE 2024), Paper No. IDETC2024-143524.
47.
Schulman
,
J.
,
Wolski
,
F.
,
Dhariwal
,
P.
,
Radford
,
A.
, and
Klimov
,
O.
,
2017
, “Proximal Policy Optimization Algorithms,” preprint arXiv:1707.06347.
You do not currently have access to this content.